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MAVEN-SA:  Model-Based  Automated  Visualization  for 
Enhanced  Situation  Awareness 

Introduction 

Identification  and  Significance  of  the  Problem 

As  the  vision  of  network-centric  warfare  (NCW,  Alberts,  Garstka,  and  Steins,  1 999) 
becomes  reality  in  the  Future  Force,  it  is  clear  that  trading  steel  for  information  represents  a  true 
revolution  in  military  affairs.  One  of  the  main  goals  of  NCW  is  to  eliminate  traditional 
information  stovepipes  to  speed  the  flow  of  information  to  the  end  user  and  to  rapidly  increase 
the  rate  at  which  information  is  transformed  to  be  militarily  significant  and  decision  centered. 
Irrespective  of  the  concept  and  goals  of  NCW,  however,  sound,  rapid,  and  accurate  decision 
making  remains  at  the  heart  of  all  aspects  of  battle  command.  To  be  truly  useful  for  decision 
making,  new  information  must  be  integrated  into  the  warfighter’s  mental  image  of  the  current 
situation  and  accurately  related  to  the  mission.  Having  a  solid  understanding  of  the  situation  is 
necessary  to  ask  the  right  questions;  having  the  right  information  is  necessary  to  answering  the 
questions  correctly  and  understanding  the  consequences  of  the  answers. 

Developing  accurate  and  sufficient  mental  models  for  battlefield  situations  and  decision 
making  is  a  necessary  prerequisite  for  understanding  the  situation  and  acting  on  key  information. 
Visualizing  situations,  information,  and  processes  is  a  useful  technique  for  developing  such 
mental  models,  understanding  how  new  information  relates  to  decisions,  and  understanding  what 
information  is  necessary  to  make  good  decisions.  This  involves  at  least  three  different  levels  of 
warfighter  knowledge  that  must  be  learned  to  achieve  and  capitalize  on  information  and 
technological  superiority:  (a)  knowledge  of  the  basic  visualization  skills  that  support  military 
decision  making,  (b)  knowledge  of  basic  visualization  tools  available  to  military  personnel,  and 
(c)  knowledge  of  how  to  apply  visualization  skills  with  the  available  tools  in  militarily  relevant 
situations. 

We  propose  to  improve  the  quality  of  military  decision  making  instruction  regarding 
visualization  of  information  by,  (a)  further  developing  the  scientific  basis  for  developing  mental 
models,  visualizing  information  and  situations,  and  teaching  visualization  techniques,  (b)  by 
developing  courseware  to  present  these  concepts  in  (a),  and  (c)  by  developing  computer-based 
instruction  software  to  facilitate  the  teaching  process  and  reduce  training  costs.  This  effort  will 
demonstrate  the  effectiveness  and  utility  of  human-computer  visualization  for  military  decision 
making  and  represent  a  significant  step  forward  in  digital  instruction  technique  and  product. 

A  technical  challenge  in  this  information  age  transformation  is  to  understand  how  various 
technological  capabilities  can  be  combined,  utilized,  and  embraced  in  a  manner  that  will 
fundamentally  improve  mission  effectiveness.  This  requires  not  only  an  understanding  of  the 
technology,  but  an  understanding  of  how  soldiers  can  best  use  that  technology,  and  how  that 
usage  can  fit  into  or  transform  military  doctrine.  At  the  core  of  military  doctrine  is  the  Military 
Decision  making  Process  (MDMP),  a  methodical,  deliberate  analytic  process  for  problem- 
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solving  that  pervades  all  military  operations.  If  transformation  is  to  truly  represent  a  revolution  in 
military  affairs,  it  must  enable  fundamental  improvements  to  the  MDMP  and  tactical  decision 
making. 

One  evolutionary  change  to  MDMP  is  the  move  towards  a  running  estimate  of 
battlespace  information  that  will  allow  more  rapid  assessment,  awareness,  and  understanding  of 
the  situation.  The  goal  of  this  change  is  to  ensure  information  superiority,  enabling  more  rapid 
decision  making,  and  result  in  more  decisive  battles.  For  example,  the  development  of  the  Global 
Information  Grid  (GIG,  Alberts  &  Hayes,  2003)  will  vastly  increase  the  amount  of  information 
available  to  all  echelons  of  command  and  will  allow  information  sharing  and  collaboration  to  be 
conducted  in  a  peer-to-peer  manner.  This  will  enable  information  to  break  beyond  the  bounds  of 
the  traditional  command  hierarchy,  in  effect,  pushing  the  power  of  information  to  the  edge  of  the 
force  network.  To  the  warfighter,  this  means  both  the  empowerment  that  more  information 
provides,  but  also  the  burden  of  making  sense  of  that  information.  Developing  the  technology 
that  will  allow  warfighters  to  rapidly  understand  and  process  large  amounts  of  dynamically 
changing  data  is  critical  to  realizing  the  NCW  vision  of  dramatically  increased  mission 
effectiveness,  self-synchronization,  improved  information  sharing  and  collaboration,  and  an 
improved,  shared  situation  awareness  (SA,  Alberts,  2002). 

Phase  I  Research 

We  proposed  to  combine  state  of  the  art  techniques  in  situation  awareness,  agent-based 
cognitive  systems,  information  visualization,  and  intelligent  instruction  interfaces  to  create  a 
system  for  teaching  visualization  skills  to  military  officers  at  the  company  command  level.  Phase 
I  demonstrated  the  feasibility  of  our  approach  by  addressing  three  main  areas  of  research: 

1 .  Models  for  SA  -  We  conducted  a  goal-directed  task  analysis  (GDTA)  of  the  cordon-and- 
search  mission  type. 

2.  Visualization  Optimization  -  We  applied  novel  visualization  tools  designed  for  scientific 
visualization  to  a  Cordon-and-Search  scenario. 

3.  Visualization  Training  Assessment  -  Researched  training  gaps  between  what  is  taught  in 
traditional  military  schoolhouse  training  and  skills  needed  in  current  military  operations. 

These  research  areas  are  addressed  in  the  context  of  developing  new  training  techniques 
and  tools  for  instruction  that  is  contextually  sensitive  to  the  warfighter’s  needs.  From  this  work 
we  have  developed  four  possible  options  for  future  development.  We  propose  to  implement 
planned  system  by  augmenting  an  existing  visualization  toolkit,  developing  a  framework  for 
implementing  instructional  scaffolding,  and  developing  coursework  that  addresses  mental  model 
development,  use  of  visualization  techniques,  and  visualizing  complex  battlefield  relationships. 
We  ground  the  problem  and  motivate  our  research  efforts  by  focusing  on  the  context  of  Military 
Operations  in  Urban  Terrain  (MOUT).  Specifically,  we  address  the  information,  decision 
making,  visualization  and  situation  awareness  needs  of  an  FCS  company  commander  conducting 
cordon  and  search  missions. 
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Background 


Decision  Making  Challenges  During  MOUT 

MOUT  operations  represent  one  of  the  most  challenging  and  dangerous  of  all  classes  of 
military  missions  because  terrain,  infrastructure,  cultural  issues,  and  other  aspects  of  the  domain 
negate  many  of  the  technological  advantages  held  by  the  U.S.  military.  Information, 
understanding  and  use  of  that  understanding  to  make  better  and  faster  tactical  decisions  is  among 
the  most  important  weapons  for  achieving  success  in  the  urban  environment.  A  frequent  type  of 
mission  in  which  company  commanders  face  numerous  decisions  is  cordon  and  search.  In  this 
type  of  operation,  commanders  are  typically  searching  for  specific  items  (e.g.,  weapons)  or 
people  (e.g.  fugitives).  The  main  tasks  for  a  cordon  and  search  are  moving  to  the  objective, 
establishing  the  cordon  to  isolate  and  secure  the  objective,  and  conducting  the  search.  While  a 
cordon  and  search  is  conceptually  straightforward,  there  are  several  key  decisions  that  must  be 
constantly  evaluated  and  many  factors  that  a  commander  must  consider.  Some  key  decisions  are: 

•  What  is  our  task  and  purpose  (mission)?  How  do  we  know  when  our  mission  has  been 
accomplished? 

•  Has  the  tactical  situation  changed?  If  so,  has  it  changed  the  mission? 

•  Is  there  an  unexpected  threat?  If  so,  what  is  its  nature? 

•  Can  I  still  complete  the  mission  with  available  resources?  If  not,  are  additional  resources 
available? 

•  Is  there  a  secure  evacuation  route  if  necessary? 

•  Each  of  these  decisions  relies  on  an  accurate  assessment  of  available  information,  an 
understanding  of  how  that  information  impacts  the  current  situation,  and  the  commander’s  ability 
to  use  that  understanding  to  accurately  predict  future  situations  that  result  in  a  successful 
mission. 

In  evaluating  these  decisions,  commanders  are  trained  to  consider  several  key  factors, 
each  of  which  evokes  its  own  set  of  questions.  Factors  for  cordon  and  search  might  be: 

•  Mission  -  What  type  of  search  is  required,  what  is  to  be  sought,  and  what  is  the  desired 
end-state?  Is  the  mission  conducted  under  surgical,  precision,  or  high-intensity  conditions? 

•  Enemy  -  What  type  of  resistance  is  to  be  expected?  How  well  are  they  armed  and 
organized?  Are  they  likely  to  be  reinforced? 

•  Terrain  &  Weather  -  What  is  the  geographic  layout?  What  types  of  buildings  are  being 
searched?  What  are  best  routes  in  and  out  each  building  and  the  area  of  operations?  What  are  the 
best  covered  and  concealed  locations  for  friendly  or  enemy  troops? 

•  Troops  Available  -  Do  I  have  enough  troops  to  accomplish  the  mission?  Do  they  have 
the  right  weapons,  equipment  and  training?  Do  they  have  enough  ammo  and  supplies? 

•  Time  Available  -  Do  we  have  time  to  be  careful  and  methodical,  or  is  speed  critical? 

•  Civil  Considerations  -  Is  the  local  population  hostile,  friendly,  or  neutral?  Has  there 
been  a  history  of  unrest  in  the  area?  Are  there  mosques,  churches,  schools,  hospitals,  or  other  no¬ 
fire  areas?  What  other  cultural  considerations  must  be  considered?  How  do  the  Rules  of 
Engagement  impact  the  mission? 
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These  key  decision  factors  are,  of  course,  interrelated  such  that  changes  in  one  factor  can 
affect  other  factors.  For  example,  any  change  to  the  mission  will  cause  almost  all  other  factors  to 
be  reevaluated.  Although  commanders  are  responsible  for  evaluating  the  key  decisions  and 
factors  constantly,  it  is  not  possible  for  them  to  deliberately  consider  each  question  constantly. 
Instead  commanders  and  other  expert  decision  makers  focus  on  changes  to  the  situation  to 
prompt  deliberate  evaluations;  this  reduces  and  simplifies  the  commanders’  decision  to  just  one: 
has  my  situation  changed?  Instead  of  evaluating  each  new  piece  of  information  according  to  a 
vast  set  of  criteria,  the  one  question  that  can  quickly  filter  irrelevant  information  is  whether  the 
new  information  changes  the  current  situation.  If  it  does,  then  the  deeper  issues  can  be  further 
explored. 

Given  enough  time,  skilled  commanders  can  typically  make  accurate  evaluations  of  new 
information  relative  to  the  current  situation.  However,  when  information  is  arriving  too  quickly 
for  proper  evaluation,  there  is  an  increased  likelihood  that  something  will  be  missed.  With  a 
predicted  order  of  magnitude  increase  in  available  information  and  the  availability  of 
unprocessed  intelligence  envisioned  by  information-age  transformation  (Alberts  &  Hayes,  2003), 
adequately  assessing  all  new  information  in  a  timely  manner  will  not  be  possible  without 
additional  fusion  and  other  automated  information  processing  to  facilitate  warfighter 
understanding.  This  problem  is  further  compounded  with  the  proliferation  of  unmanned  and 
unattended  sensors  being  developed  as  part  of  the  Future  Combat  Systems  (FCS),  especially 
those  organic  to  the  FCS  company,  because  that  information  will  most  certainly  be  unprocessed 
by  higher  echelon  intelligence  staff.  Furthermore,  much  information  that  is  critical  for  urban 
operations  is  cultural  or  otherwise  non-physical  in  nature.  Given  the  challenges  of  large  volumes 
of  information  that  can  be  both  uncertain  and  unprocessed,  how  can  future  commanders  make 
use  of  this  new  information  to  better  understand  their  situation  and  the  tactical  decisions  that 
must  be  made? 

Situation  Awareness  to  Support  Decision  Making 

Situation  awareness  forms  the  foundation  for  military  decision  making  and  task 
execution.  Soldiers  must  do  more  than  simply  perceive  the  state  of  their  environment.  They  must 
understand  the  integrated  meaning  of  what  they  perceive  in  relation  to  their  goals.  SA,  as  such, 
incorporates  the  soldiers’  understanding  of  the  situation  as  a  whole  and  forms  a  basis  for  decision 
making.  How  they  identify  their  options  for  action  is  a  function  of  their  situation  awareness  -  the 
understanding  of  what  has  happened  and  of  the  resulting  battlefield  situation.  Their  SA  provides 
the  context  in  which  the  decision  takes  place.  Battlefield  visualization  displays  must  therefore 
provide  warfighters  with  adequate  context  to  allow  them  to  gain  and  maintain  the  SA  they  need 
to  make  effective  decisions.  In  the  demanding  combat  environment,  superior  SA  can  bring 
tremendous  advantages  by  promoting  information  dominance,  improving  security  and 
survivability,  and  optimizing  lethality. 

Situation  Awareness  is  formally  defined  as  “the  perception  of  the  elements  in  the 
environment,  the  comprehension  of  their  meaning  and  the  projection  of  their  status  in  the  near 
future”  (Endsley,  1988).  Situation  awareness  therefore  involves  perceiving  critical  factors  in  the 
environment  (Level  1  SA);  understanding  what  those  factors  mean,  particularly  when  integrated 
together  in  relation  to  the  warfighter's  goals  (Level  2  SA);  and  at  the  highest  level,  an 


4 


understanding  of  what  the  situation  will  be  in  the  near  future  (Level  3  SA).  These  two  higher 
levels  of  SA  allow  warfighters  to  function  in  a  timely  and  effective  manner. 

Similarly,  the  Army  has  operationally  defined  situational  awareness  as  “the  degree  to 
which  one  is  able  to  maintain  a  common  operating  picture  of  all  aspects  of  the  tactical  situation” 
(Department  of  the  Army,  2002a).  This  definition  corresponds  closely  to  Endsley’s  Level  2  SA 
and  at  the  company  level  is  measured  by  answers  to  the  following  questions: 

•  Where  am  I  (relative  to  where  I’m  supposed  to  be)? 

•  Where  my  soldiers  and  what  are  is  their  status? 

•  Where  are  the  friendly  forces  and  what  is  their  status? 

•  Where  are  the  enemy  and  what  is  its  capabilities? 

One  experimental  technique  for  reacquiring  SA  during  a  mission  is  for  the  company 
commander  to  issue  the  “Go  Firm  command”.  This  command  requires  subordinate  units  to  pause 
current  operations,  assume  a  hasty  defensive  posture  and  provide  situation  reports  to  the 
commander.  With  advances  in  blue-force  tracking  technology,  much  of  this  information  may 
eventually  be  provided  automatically,  however,  the  commander  still  needs  to  process  that 
information  and  understand  how  it  affects  his1  assessment  of  the  situation.  The  further  challenge 
is  how  to  present  this  information  in  a  way  that  does  not  unduly  increase  the  commander’s 
cognitive  workload  and  doesn’t  limit  the  degree  to  which  the  commander  can  sense  other 
information  in  the  immediate  environment.  Graphical  and  other  low-workload  means  for 
presenting  information  to  the  commander  are  essential  for  enhancing  the  commander’s  ability  to 
achieve  and  maintain  high  levels  of  SA.  Given  that  most  battlefield  visualizations  are  map-based, 
graphical  information  presentation  tools  must  integrate  seamlessly  within  that  view. 

Visualization  Skills  Training  to  Support  Decision  Making 

Battlefield  visualization  is  defined  as  “the  mental  process  which  supports  the 
commander’s  decision  making  process”  (Department  of  the  Army,  1995).  It  “lies  at  the  center  of 
battle  command”  and  is  “an  essential  leadership  attribute  of  the  commander”  that  is  “critical  for 
accomplishing  missions.”  Battlefield  visualization  is  a  three-step  command  process  whereby  the 
commander  develops  a  clear  understanding  of  the  current  situation,  envisions  a  desired  end  state, 
and  visualizes  the  sequences  of  activity  that  will  move  his  force  from  its  current  situation  to  the 
desired  end  state  (Department  of  the  Army,  2002b).  Thus  battlefield  visualization  is  an 
inherently  human  activity  that  corresponds  directly  to  Level  3  SA  and  informs  tactical  decision 
making. 

If  a  new  training  approach  is  to  be  developed  to  improve  tactical  decision  making,  it  must 
address  battlefield  visualization  and,  more  specifically,  situation  awareness.  This  approach 
requires  a  focus  on  basic  visualization  skills  including  identification  of  information  needs  and 
sources;  integration  of  these  sources  in  external  visual  representations,  such  as  maps  and  tables; 
the  drawing  of  valid  inferences  from  these  representations;  and  the  integration  of  new 


1  According  to  current  military  doctrine,  all  members  of  the  U.S.  Army  Infantry  are  male,  and  hence  they 
are  referred  to  using  only  masculine  pronouns. 
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information  into  existing  knowledge  schemas.  This  approach  will  need  to  draw  on  flexibility  of 
contemporary  digital  displays,  the  personalized  instruction  of  advanced  computer-based  training 
systems,  and  current  advances  in  situation  awareness  and  cognitive  psychology  research. 


Phase  I  Technical  Objectives  and  Approach 

The  primary  goal  of  this  project  is  to  explore  how  to  use  educational  technology  to 
support  battlefield  visualization  training.  Our  goal  was  to  identify  opportunities  to  introduce  the 
concept  of  battlefield  visualization  training  in  to  the  military  schoolhouse  and  to  improve 
battlefield  visualization  through  the  use  of  innovative  computer  based  training.  The  result  of  this 
improved  training  should  fundamentally  improve  the  battlefield  commanders’  ability  to  acquire 
SA  and  make  decisions.  We  addressed  this  training  goal  in  the  context  of  FCS  company-level 
urban  operations.  To  narrow  the  scope  of  this  effort  we  will  focus  on  cordon  and  search 
operations,  a  high-frequency  mission  type  where  timely  information,  accurate  situation 
awareness,  and  rapid  decision  making  are  especially  critical  to  success. 

The  challenges  for  accomplishing  this  goal  were  divided  into  three  core  areas.  These 
areas  corresponded  to  the  main  responsibilities  and  core  competences  of  each  MAVEN-SA  team 
member: 

1 .  Understanding  the  warfighter’s  needs  and  the  context  of  tactical  decision  making  (SA 
Technologies). 

2.  Develop  approach  for  battlefield  visualization  training  including  identifying  basic 
visualization  skills,  training  methodology,  types  of  materials,  and  core  concepts  for  intelligent 
computer-based  training.  (Soar  Technology). 

3.  Creating  innovative  information  visualization  and  mixed-initiative  interaction  techniques 
that  support  that  battlefield  visualization.  (NCSU). 

The  project  focused  on  answering  several  feasibility  questions,  including: 

1 .  Can  specific  current  skill  gaps  between  battlefield  visualization  schoolhouse  training  and 
visualization  skills  required  by  current  military  operations  be  identified? 

2.  Can  basic,  trainable,  visualization  skills  be  identified  that  fill  these  gaps? 

3.  Can  an  educational  technology  approach,  including  specific  training  modules  and  software 
systems,  be  developed  that  would  produce  measurable  improvements  in  these  gap  areas? 

4.  Would  such  an  approach  actually  improve  warfighter  performance? 

The  following  sections  describe  the  initial  vision  for  how  the  proposed  training 
methodology  and  materials  might  be  applied,  the  context  of  how  they  might  be  incorporated  with 
other  decision  and  performance  aids,  and  their  technical  and  scientific  basis. 
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The  Vision:  Enabling  Tactical  Battlefield  Commanders  Through  Improved  Visualization 
Skills 


As  discussed  earlier,  the  key  risk  of 
urban  operations  is  that  the  urban 
environment  can  obscure  intelligence  about 
the  enemy,  their  strength,  and  intent  until  it 
is  too  late.  The  goal  is  to  provide  training 
that  will  improve  the  battle  commander’s 
ability  to  visualize  the  current  situation 
based  on  the  information  he  has  and 
visualize  the  meaning  of  this  information  so 
he  can  fully  comprehend  its  implications 
and  visualize  the  future  status  of  the 
operation  environment  so  he  can  maintain 
control  of  the  tactical  situation.  Generally, 
the  creation  of  these  visualizations  can  be 
facilitated  by  the  processes  of  pattern 
matching  or  the  use  of  multi-dimensional 
visual  overlays.  Examples  of  pattern 
matching  include  visualizing  the  patterns  of  obstacles  or  explosive  devices  that  might  indicate 
high-value  buildings  or  other  targets,  or  might,  instead,  indicate  an  effort  to  canalize  friendly 
forces  into  a  kill  zone.  Other  visualization  patterns  include  cell  phone  traffic,  reconnaissance 
elements,  or  civilian  activity  that  might  indicate  an  ambush  or  attack.  More  obvious  patterns 
might  include  red-force  tracking  that  could  indicate  direction  and  strength  of  an  attack.  While 
technology  exists  for  visualizing  patterns  within  a  single,  homogeneous  data  type,  the  challenge 
remains  for  finding  and  presenting  data  and  then  training  leaders  to  see  patterns  across  data  types 
(e.g.,  the  combination  of  explosive  devices  and  cell  phone  traffic). 

Visualizing  multiple  dimensions  of  the  urban  environment  is  another  important  aspect  of 
achieving  and  maintaining  high  levels  of  situation  awareness.  How  can  we  visualize  avenues  of 
approach,  dynamically  calculated  evacuation  routes,  subterranean  infrastructure,  uncertainty  and 
reliability  of  information,  cultural  aspects  of  the  area,  and  the  myriad  other  details  that 
commanders  today  must  try  to  keep  in  their  heads?  Combining  structural,  mission,  intelligence, 
and  cultural  information  can  be  overwhelming;  training  commanders  to  understand  the  critical 
elements  of  each  domain  and  to  leverage  visual  representations  to  better  convey  the  situation  are 
additional  challenges  we  needed  to  address. 

Phase  I  Activities  and  Accomplishments 

Under  Phase  I  of  this  Small  Business  Technology  Transfer  Research  contract,  we 
researched  the  use  of  educational  technology  to  support  battlefield  visualization  training.  Our 
goal  was  to  identify  opportunities  to  introduce  the  concept  of  battlefield  visualization  training  in 
to  the  military  schoolhouse  and  to  improve  battlefield  visualization  through  the  use  of  innovative 
computer  based  training.  Specifically,  we  wanted  to  develop  a  training  capability  that  would 
provide  significant  improvement  to  a  commanders’  ability  to  use  visual  representations  and 


Figure  1.  Dynamic  information  overlaid  on  urban  map 
reflects  ingress  and  evacuation  routes,  friendly  and  enemy 
forces,  friendly  and  hostile  neighborhoods,  no-fire  areas, 
and  other  mission  specific  information. 
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displays  to  assemble  and  integrate  heterogeneous  information  sources  to  maintain  situation 
awareness,  improve  planning  and  aid  communication. 

During  the  initial  phase  of  this  project  we  progressed  in  several  areas.  Major 
accomplishments  of  Phase  I  included: 

•  A  working  scenario  was  defined  on  which  to  determine  project  feasibility.  This  scenario 
centered  on  a  company  level  MOUT  cordon  and  search  mission. 

•  A  comprehensive  GDTA  was  performed  to  identify  the  information  required  by  a 
company  commander  to  maintain  SA  during  a  MOUT  Cordon  and  search  mission. 

•  Reviewed  appropriate  literatures  describing: 

o  Military  battlespace  visualization 

o  Cognitive  and  perceptual  psychology  and  linguistics  as  they  relate  to  visualization, 
visual  languages,  and  external  knowledge  representations 

o  Computer  science  as  it  relates  to  visualization,  visual  languages,  and  mixed- 
initiative  interaction. 

o  Education,  cognitive  and  social  psychology,  and  computer  science  as  they  relate  to 
general  approaches  to  educational  technology. 

•  Developed  a  basic  visualization  capability  for  perceptual  display  optimization  for 
MOUT  environments,  transitioning  our  STTR  academic  partners  previous  work  in  perception 
optimized  visualization  and  mixed-interaction  interfaces.  This  display  and  interaction  capability 
will  provide  the  basis  for  our  intelligent  part-task  visualization  trainer. 

•  Developed  concrete  phase  II  training  goals  and  training  tool/environment  approaches. 
MOUT  Scenario:  Company  Level  Cordon  &  Search 

As  previously  noted,  MOUT  operations  represent  one  of  the  most  challenging  and 
dangerous  of  all  classes  of  military  missions.  Information,  understanding  and  use  of  that 
understanding  to  make  better  and  faster  tactical  decisions  are  among  the  most  important  weapons 
for  achieving  success  in  the  urban  environment.  A  frequent  mission  type  in  which  company 
commanders  face  numerous  decisions  is  cordon  and  search.  During  Phase  I,  we  developed  a 
detailed  description  and  analysis  of  a  cordon  and  search  scenario. 
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Situation:  You  are  the  Commander  of  A  Company,  1st  Battalion,  30th  Infantry.  You  are 
part  of  TF  Saber,  a  brigade  task  force  deployed  to  Taronia  to  assist  the  government  and 
provide  security  and  stability  operations  until  host  nation  forces  are  able  to  secure  the 
region  from  insurgent  forces. 

Enemy  Forces:  Insurgent  activity  in  the  Taronian  capital  of  Brovburg  has  escalated  over  the  last  two 
weeks.  The  Brigade  Intelligence  Estimate  points  to  evidence  that  insurgents  have  a  larger  than 
expected  amount  of  explosives  and  weapons  and  are  planning  more  aggressive  attacks  in  order  to 
destabilize  the  Taronian  government.  Recent  insurgent  activity  has  included  2-3  man  rocket  propelled 
grenade  (RPG)  teams  attacks,  isolated  car  bombs  and  hit  and  run  mortar  attacks  on  coalition  and  host 
nation  forces.  The  town  of  McKenna,  the  largest  suburb  of  Brovburg,  is  suspected  of  harboring  a 
growing  cell  of  insurgents  with  multiple  caches  of  weapons  and  explosives.  Insurgents  in  McKenna  are 
often  reinforced  from  the  surrounding  villages  of  Luton  and  Carona,  west  and  east  of  McKenna. 
Activity  in  McKenna  has  been  excessively  violent  over  the  last  72  hours.  Insurgents  have  launched 
multiple  attacks,  both  day  and  night,  against  U.S.  forces  and  against  Taronian  militia  attempting  to 
restore  order.  These  insurgent  forces  are  capable  of  squad  size  attacks  and  typically  employ  ambushes, 
sniper  attacks,  RPG  teams  and  Improvised  Explosive  Devices  (DEDs)  to  disrupt  security  operations. 
Insurgents  are  likely  to  attack  isolated  elements  and  convoys,  and  then  break  contact  to  prevent  an  all 
out  engagement.  The  insurgents  then  dissipate  back  into  the  general  population  and  are  not  easily 
identified. 

Friendly  Forces:  Higher  Headquarters:  1st  Battalion,  30th  Infantry:  NLT  17  1200  March,  1st 
Battalion,  30*  Infantry  conducts  cordon  and  search  and  secures  McKenna  in  order  to  allow  Taronian 
forces  to  restore  regional  stability.  NLT  17  1200  March,  A  Company,  the  battalion  main  effort, 
conducts  a  cordon  and  search  and  secures  McKenna  in  order  to  allow  Taronian  forces  to  restore 
regional  stability.  NLT  17  1200  March,  B  Company  secures  Luton  to  prevent  insurgents  from 
disrupting  A  Company's  cordon  and  search.  NLT  1 7  1200  March,  C  Company  secures  Carona  in  order 
to  prevent  insurgents  from  disrupting  A  Company's  cordon  and  search.  NLT  17  1200  March,  D 
Company  secures  roadblocks  on  Highway  1 6  and  3 1  in  order  to  prevent  insurgents  from  disrupting  A 
Company's  cordon  and  search. 

Battalion  Commander’s  Intent:  The  Battalion  Commander’s  intent  is  to  isolate  the  suburb  of 
McKenna  from  Brovburg  and  the  surrounding  villages.  Once  isolated,  McKenna  is  searched 
thoroughly  and  insurgent  forces  brought  under  control.  Weapon  caches  are  located,  confiscated  or 
destroyed  and  suspected  insurgents  are  arrested  without  creating  a  hostile  environment  leading  to  direct 
firefights.  At  the  end  of  the  operation,  McKenna  is  secure  and  clear  of  insurgents  and  their  weapons. 

Mission:  NLT  17  1200  March,  A  Company  conducts  a  cordon  and  search  and  secures  McKenna  in 
order  to  allow  Taronian  forces  to  restore  regional  stability. 

Scheme  of  Maneuver:  Area  of  Operations  (AO)  McKenna,  as  shown  in  Figure  3,  is  divided  up  into 
three  different  zones:  Block  Zebra,  Block  X-ray  and  Block  Yankee.  One  platoon,  the  security  element, 
will  establish  checkpoints  and  cordon  off  the  town.  Taronian  militia  forces  will  assist  the  security 
element  in  augmenting  checkpoint  operations,  One  platoon,  the  search  element  will  conduct  the  actual 
building-to-building  search,  by  block  area.  One  platoon,  the  reserve  element,  will  stand  by  to  execute 
on  order  tasks  to  assist  either  the  security  or  search  element  in  their  tasks  or  to  conduct  other  tasks  as 
designated  by  the  company  commander.  The  company  HQ  element  will  move  with  the  search  element. 

Figure  2.  Cordon  and  search  tactical  situation. 
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Figure  3.  McKenna  map  with  block  boundaries. 

This  scenario  implies  the  following  mission  considerations  for  a  company  commander: 

1 .  Watch  out  for  snipers.  When  coalition  forces  are  in  the  vicinity  of  weapons  caches, 
insurgents  have  been  known  to  use  snipers  to  divert  attention. 

2.  Insurgents  are  excellent  at  “playing  along”  and  appearing  friendly  to  coalition  forces  in 
order  to  prevent  detection.  Keep  your  guard  up. 

3.  Vehicle  traffic  in  Block  Zebra  can  be  heavy  around  festival  time.  Insurgents  have  taken 
advantage  of  this  traffic  pattern  to  execute  car  bombs  with  both  friendly  and  civilian  casualties. 

4.  Ingress  and  egress  routes  are  critical  to  prevent  platoons  or  squads  from  being  cut  off  and 
isolated  to  insurgent  attacks. 

5.  Angry  crowds  have  quickly  formed  on  previous  cordon  and  search  operations.  Have  a  plan 
for  crowd  control  in  case  things  get  out  of  hand. 

Subsequent  sections  address  in  more  detail  two  of  the  five  mission  considerations. 

MOUT  visualization  for  sniper  threat  When  planning  to  conduct  a  cordon  and  search 
operation  in  an  urban  environment,  the  likelihood  of  snipers  or  small  2-3  man  enemy  teams 
engaging  the  cordon  and  search  force  is  high  in  a  hostile  neighborhood.  Insurgents  may  attempt 
to  disrupt,  delay  or  prevent  a  successful  operation,  especially  if  there  are  weapons  or  contraband 
caches  that  are  likely  to  be  found  and  lost  to  friendly  forces.  Commanders  must  determine  the 
impact  of  this  threat  and  take  measures  to  actively  prevent  sniper  attacks  from  occurring. 

In  planning  such  an  operation,  the  commander  needs  to  be  able  to  visualize  the  battlefield. 
Looking  at  the  results  of  the  GDTA  conducted  for  a  cordon  and  search  operation,  we  will  show 
how  improving  the  commander's  visualization  would  support  the  information  requirements 
needed  for  meeting  the  commander's  cordon  and  search  goals. 

The  GDTA  goal  ‘Avoid  Danger  Areas’  addresses  several  areas  where  the  commander 
needs  to  visualize  enemy  activity  in  the  AO.  Visualizing  the  location  of  neighborhoods  or 
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residences  hostile  to  friendly  forces  allows  him  to  better  predict  where  contact  might  occur. 
Visualizing  likely  sniper  positions  and  the  fields  of  fires  and  observations  from  rooftops, 
doorways  and  windows  allow  the  commander  to  view  potential  areas  to  cover  with  counter 
sniper  teams  or  to  orient  covering  fires  toward  various  vantage  points  that  a  sniper  might  occupy. 

A  thorough  assessment  of  the  following  is  critical  to  mission  success: 

•  Types  of  buildings  in  and  adjacent  to  the  AO 

•  Types  and  locations  of  building  rooftops,  windows  and  doorways 

•  Fields  of  Fire  from  rooftops,  windows  and  doorways 

•  Observation  from  rooftops,  windows  and  doorways 

•  Routes  of  ingress  and  egress  to  the  objective  area  for  friendly  forces 

•  Most  likely  escape  routes  for  small  enemy  elements  out  of  the  objective  area 

•  Locations  of  security,  search  and  reserve  elements 

•  Likely  enemy  sniper  positions 

•  Attitude  of  civilians  in  the  objective  neighborhood 

•  Evidence  of  past  sniper  activity 

•  Exposed  areas  where  friendly  forces  are  vulnerable 

•  Cover  and/or  concealment  from  urban  clutter 

Figure  4  shows  the  area  with  likely  sniper  locations  identified.  At  this  stage,  a 
visualization  training  tool  could  aid  the  commander  in  learning  how  to  identify  potential  danger 
areas  by  considering  the  fields  of  observation  and  fields  of  fire  from  these  locations,  as  well  as 
the  effective  weapons  ranges  of  likely  enemy  weapons  in  those  locations.  Subsequent 
assessment  could  identify  areas  with  suitable  cover  where  friendly  forces  could  be  positioned  to 
address  these  threats. 


Figure  4.  McKenna  map  with  likely  sniper  locations. 

MOUT  visualization  of  ingress  and  egress  routes.  One  difficult  aspect  of  urban 
operations  is  the  multitude  of  possible  avenues  of  approach  and  escape  in  and  around  streets, 
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alleyways,  buildings  and  urban  clutter.  For  a  successful  mission,  the  commander  must  clearly 
identify  these  areas  and  evaluate  their  potential  advantage  and  disadvantage  from  both  the  enemy 
and  the  friendly  perspective. 

Two  major  goals  for  planning  cordon  and  search  operations  include  ‘Project  Enemy 
Behavior,’  and  ‘Plan  Route.’  Satisfying  these  goals  provides  the  commander  with  the  battlefield 
situation  that  he  must  visualize  prior  to  moving  into  the  AO  to  conduct  the  cordon  and  search. 
Identifying  routes  of  ingress  and  egress  among  the  buildings  as  well  as  building  entry  and  exit 
points  allow  him  to  plan  on  possible  use  of  roadblocks  or  security  team  locations  to  prevent 
insurgents  from  escaping  the  area  and  to  aid  his  forces  entry  and  withdrawal  from  the  area.  The 
commander  will  place  his  reserve  element  where  they  can  quickly  maneuver  along  a  route  to 
accomplish  a  task,  for  example,  to  reinforce  a  vehicle  checkpoint.  Visualizing  where  vehicle  or 
pedestrian  traffic  may  impact  his  mission  will  help  the  commander  see  where  security  element 
positions  are  best  employed.  He  can  also  identify  possible  locations  where  civilian  crowds  may 
gather  and  attempt  to  disrupt  or  influence  the  search. 

Urban  areas  present  several  aspects  for  careful  consideration  including  the  following: 

•  Building  entry  and  exit  points 

•  Types  of  streets  and  alleyways  between  buildings 

•  Location  of  open  areas  such  as  parks,  markets,  public  squares,  and  athletic  fields 

•  Subterranean  features  such  as  irrigation  channels  or  sewer  systems 

•  Building  and  block  patterns 

•  Surrounding  terrain  features  such  as  wooded  areas  or  water  obstacles 

•  Observation  and  fields  of  fire  from  various  vantage  points,  both  inside  of  buildings  and 
outside  on  streets 

•  Funnel  areas  or  choke  points  that  inhibit  dispersion 

•  Conditions  of  streets  or  roads  that  might  impact  vehicle  speed  and  traffic  flow 

•  Cover  and  concealment  along  routes  of  ingress  and  egress 

Using  a  visualization  tool,  the  commander  can  learn  to  determine  where  he  should 
emplace  his  forces  (see  Figure  5).  With  security  augmentation  from  host  nation  militia,  the 
commander  can  focus  his  forces  on  the  most  critical  areas  in  his  AO  and  assign  missions  to  each 
element  in  force.  Once  he  identifies  the  ingress  and  egress  routes  that  would  spill  out  of  or 
originate  in  his  sector,  the  commander  can  determine  that  the  optimal  places  to  place  his  security 
element  is  at  check  points  (CP)  1,3,  and  5.  He  knows  he  can  then  assign  CP  2,  4,  and  6  and  the 
wooded  area  between  CPs  2  and  3  to  the  host  nation  militia  force  assisting  him.  At  the 
checkpoints,  the  squads  would  control  vehicle  and  pedestrian  traffic  to  cordon  off  the  area  and 
prevent  disruption  to  the  search  platoon.  He  can  brief  the  search  element  and  the  reserve  on  the 
security  plan  and  ensure  that  they  maintain  awareness  of  these  ingress  and  egress  routes  so  that 
they  can  react  and  stop  possible  escapes  from  insurgents.  He  can  orient  his  security  forces  down 
streets  or  alleyways  and  to  cover  sewer  points  of  entry  as  well. 
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Figure  5.  McKenna  map  with  ingress  and  egress  routes,  and  troop  placement. 

Combining  knowledge  of  ingress  and  egress  routes  for  insurgents  with  the  sniper 
information,  training  will  support  the  commander  in  visualizing  the  placement  of  his  forces  in 
relation  to  likely  enemy  sniper  positions.  In  an  urban  environment  (see  Figure  6),  he  will  notice 
that  snipers  from  the  other  block  areas  can  easily  influence  his  AO.  The  training  tool  will 
sensitize  him  to  areas  of  possible  contact;  ensuring  they  are  observed  by  search  and  security 
elements.  The  commander  will  see  the  position  of  his  forces  and  determine  what  elements  might 
be  more  exposed  to  sniper  fire.  He  can  then  determine  how  best  to  use  available  block,  building 
and  street  patterns  to  mask  movement  without  exposing  his  forces  unnecessarily  to  enemy 
threats.  The  visualization  training  tool  will  also  enable  him  to  see  where  to  emplace  counter¬ 
sniper  teams  and  where  to  orient  his  platoon  leaders  observation  and  fields  of  fire.  The 
commander  can  see  possible  locations  to  position  his  reserve  element  where  they  can  be  the  most 
responsive  to  his  operation  without  being  at  undue  risk. 
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Figure  6.  McKenna  map  with  ingress  and  egress  routes,  likely  sniper  locations,  and  troop  placement. 


MOUT  Scenario:  Situation  Awareness  Requirements 

SA  is  domain-specific;  by  this  we  mean  that  the  knowledge  and  skills  necessary  to 
develop  superior  SA  in  one  domain  may  be  generally  very  different  from  those  in  another.  For 
example,  the  knowledge  and  skills  needed  for  a  Company  commander  to  develop  good  S A  are 
significantly  different  from  the  skills  and  knowledge  a  commercial  airline  pilot  would  need  to 
develop  good  SA,  while  dismount  infantry  in  combat  and  cordon  and  search  units  use  very 
related  knowledge  and  skills.  Not  only  are  the  tools  used  by  the  individuals  vastly  different,  but 
so  are  the  information  gathering  techniques,  the  skills  required,  and  the  knowledge  bases  and 
mental  models  that  provide  a  framework  for  the  development  of  SA.  The  first  step,  then,  in 
developing  a  training  program  to  improve  SA  within  a  domain,  such  as  dismounted  infantry,  is  to 
conduct  some  form  of  requirements  analysis  to  identify  the  domain-specific  dynamic  information 
requirements  (i.e.,  the  SA  needs)  for  the  position  of  interest  (e.g.,  the  Company  commander.) 

An  SA  requirements  analysis  usually  employs  a  combination  of  cognitive  engineering 
procedures.  Expert  elicitation,  observation  of  operator  performance  of  tasks,  verbal  protocols, 
formal  questionnaires  and  analysis  of  written  materials  and  documentation,  i.e.  training  manuals 
and  checklists,  have  formed  the  basis  for  these  analyses.  To  date,  such  analyses  have  been 
completed  for  many  domains  including  fighter  pilots  Endsley,  1993,  bomber  pilots  (Endsley, 
1989),  TRACON  air  traffic  control  (Endsley  &  Jones,  1995),  infantry  platoon  leaders  (Strater, 
Endsley,  Pleban,  &  Matthews,  2000),  Army  brigade  level  officers  (Bolstad,  Riley,  Jones,  & 
Endsley,  2002),  and  Naval  air  warfare  coordinators  (Strater,  Endsley,  &  Plott,  2004).  A  similar 
process  has  been  employed  by  Hogg,  Torralba  and  Yolden  (1993)  to  determine  appropriate 
queries  for  a  nuclear  reactor  domain. 
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Frequently,  the  problem  of  determining  what  aspects  of  the  situation  are  important  for  a 
particular  operator’s  SA  has  been  approached  using  a  form  of  cognitive  task  analysis  called  a 
Goal-Directed  Task  Analysis  (GDTA),  illustrated  in  Figure  7.  In  such  an  analysis,  the  major 
goals  of  a  particular  position  are  identified,  along  with  subgoals  necessary  for  meeting  each  goal. 
An  example  of  a  goal  would  be  to  identify  and  monitor  threats,  while  a  subgoal  of  this  might  be 
communicating  with  team  members.  For  each  subgoal,  the  major  decisions  to  be  made  are 
identified.  The  SA  needs  for  making  these  decisions  and  carrying  out  each  subgoal  are 
subsequently  determined.  By  focusing  on  goals,  rather  than  tasks,  this  methodology  seeks  to 
identify  the  information  needs  directly,  without  considering  how  the  operator  will  acquire  the 
information,  as  the  method  may  change  from  one  operator  to  another,  or  as  new  technologies  are 
fielded.  In  addition,  goals  form  the  basis  for  decision  making  in  many  complex  environments. 
Finally,  the  GDTA  seeks  to  determine  the  ideal  information  needs  of  the  operator,  everything  the 
operator  would  like  to  know,  rather  than  just  the  information  that  can  be  gained  with  existing 
technologies  and  information  sources. 


Figure  7.  GDTA  goal  hierarchy  format. 

To  conduct  the  GDTA  for  the  initial  phase  of  this  investigation,  researchers  met 
individually  with  a  single  Subject  Matter  Expert  (SME),  a  skilled  and  articulate  Lieutenant 
Colonel  currently  in  the  U.S.  Army  Reserves  as  a  staff  officer  instructor.  Our  SME  is  a  military 
analyst  with  15  years  of  U.S.  Army  infantry  officer  experience,  who  served  in  multiple  positions 
from  infantry  platoon  leader  and  company  commander  to  battalion  and  brigade  staff  officer  and 
graduated  from  the  Army  Command  and  General  Staff  College  at  Fort  Leavenworth,  KS.  He  is 
also  experienced  in  conducting  GDTAs  for  enhancing  SA.  During  three  interviews  of 
approximately  two  hours  each,  the  SME  responded  to  a  series  of  open-ended  questions  designed 
to  elicit  lengthy  responses  concerning  his  major  goals  and  the  decisions  associated  with  the 
accomplishment  of  these  goals  for  a  company  commander  during  a  cordon  and  search  mission  in 
urban  terrain.  He  then  addressed  the  information  needed  to  formulate  these  decisions. 
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Interviewers  held  their  questions  until  a  natural  stop  in  the  dialogue  so  as  not  to  interrupt 
the  flow  of  ideas.  When  a  natural  pause  occurred  in  the  narration,  particular  attention  was  taken 
to  ascertain  exactly  how  each  piece  of  information  is  used.  Higher-level  assessments  of  the 
situation  related  to  comprehension  and  projection  were  determined  in  this  manner.  The 
completed  analysis  was  structured  into  a  graphical  depiction  for  SME  review  and  refinement. 

For  this  GDTA,  the  SME  and  researchers  then  compared  the  results  of  the  analysis  to  other 
GDTA  information  hierarchies  from  related  domains  (i.e.  an  infantry  platoon  leader  on  a  MOUT 
mission)  to  identify  any  gaps  in  the  results. 

The  overview  page  of  the  Company  commander  analysis  shows  the  overall  goal 
hierarchy  structure,  with  Secure  and  search  assigned  area  as  the  overarching  objective  (Figure 
8).  This  is  then  broken  down  into  seven  primary  goals,  with  secondary  goals  under  each  that  may 
be  employed  to  meet  the  mission  objectives.  Each  secondary  goal  is  later  listed  on  a  separate 
page  in  the  requirements  analysis,  broken  down  into  subgoals.  For  primary  goal  1.0,  Protect  the 
force/Avoid  casualties,  five  secondary  goals  are  listed,  with  three  of  those  being  further  divided 
into  subgoals  (Figure  9).  For  each  subgoal,  some  of  the  questions  the  company  commander  is 
considering  are  listed,  followed  by  the  SA  elements  necessary  to  answer  these  questions.  Figure 
10  illustrates  the  SA  requirements  for  subgoal  1.1.2  -Avoid  danger  areas.  Listed  beneath  the 
subgoal  are  some  of  the  questions  that  the  Company  commander  will  consider:  What  is  the  least 
exposed  position  or  avenue  of  approach,  and  how  do  I  minimize  the  impact  of  danger  areas? 
Under  these  questions  is  a  list  of  information  that  the  Company  commander  will  consider  in 
addressing  this  goal,  such  as  projected  enemy  behavior,  areas  of  cover  and  concealment,  time 
constraints  and  funnel  areas. 
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Figure  8.  Overall  goal  hierarchy  for  cordon  and  search  mission. 
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Figure  9.  Primary  goal  1.0  -  Protect  the  force/ Avoid  casualties. 
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Figure  10. 1.1.2  Avoid  Danger  Areas. 


To  aid  in  understanding  the  information  needs,  an  additional  SA  requirements  analysis 
was  undertaken  to  determine  the  Level  1  (data).  Level  2  (comprehension)  and  Level  3 
(projection)  SA  needs  of  the  Company  commander.  Much  of  the  information  was  redundant 
across  a  range  of  goals,  and  this  analysis  provides  a  vehicle  for  information  consolidation.  The 
SA  requirements  were  divided  into  categories  for  threat,  friendly,  civilian,  supplies,  mission, 
terrain,  weapons,  communications,  weather,  and  MOUT-specific  information.  Table  1  shows 
how  the  information  from  subgoal  1.1.2  ‘avoid  danger  areas’  was  reorganized  in  this  SA 
analysis.  This  table  shows  that  for  this  subgoal,  supplies  and  communications  are  not  areas  of 


19 


primary  concern,  as  no  items  specific  to  those  two  categories  are  listed.  Moreover,  this  analysis 
reveals  that  many  of  the  SA  requirements  identified  in  the  GDTA  focus  on  the  higher-level 
assessments  involved  in  understanding  and  projecting  into  the  future.  Lower-level  data  needs  are 
often  left  out  of  the  goal  hierarchy  for  the  sake  of  parsimony,  however,  this  does  not  suggest  that 
the  lower-level  information  is  not  important,  but  rather  that  these  data  are  useful  to  the  extent 
that  they  support  these  assessments  that  are  critical  for  timely  decision  making  and  action.  Thus, 
the  SA  requirements  analysis  provides  a  more  complete  listing  of  the  lower  level  SA  information 
that  support  development  of  the  higher  level  assessments  necessary  for  meeting  the  Company 
commander’s  goals.  In  conjunction  the  GDTA  and  the  SA  requirements  analysis  provide  the 
research  basis  to  support  the  development  of  training  programs  targeted  to  improve  the 
visualization  skills  to  promote  the  development  of  superior  SA. 
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Table  1.  Situation  awareness  requirements  for  subgoal  1.1.2 -Avoid  danger  areas 


Level  1  SA 

Level  2  SA  ■  ''1;:  V 

Level  3  SA 

Threat 

likely  ambush  sites 

likely  sniper  positions 

projected  enemy  actions 
projected  impact  of 
contact 

Friendly 

friendly  fields  of  observation 

projected  ability  to  avoid 
fires 

projected  availability  of 
cover/ concealment 
projected  time  to 
cover/ concealment 

Civilian 

civilian  attitude 

traffic  patterns 

neighborhoods  friendly/hostile  to 

US  forces 

Supplies 

Mission 

time  constraints 

Terrain 

type  of  terrain 

buildings 

areas  of  cover 

areas  of  concealment 
funnel  areas 

level  of  exposure  on  route 
avenues  of  approach 
speed  of  movement 
likely  ambush  sites 
likely  sniper  positions 

projected  availability  of 
cover/ concealment 
projected  time  to 
cover/concealment 

Weapons 

projected  ability  to  avoid 
fires 

Comms 

Weather 

impact  of  weather 

MOUT 

parks/recreation 

areas 

location  of 
markets 

location  of  public 
squares 
location  of 
athletic  fields 
avenues  of 
approach 
cover  & 
concealment 
types  of  rooftops 
block  patterns 
street  patterns 
pedestrian 
traffic 

civilian  hostile 
attitude 
civilian  friendly 
attitude 

danger  areas/exposure  areas 

areas  of  cover 

areas  of  concealment 

funnel  areas 

level  of  exposure  on  route 

avenues  of  approach 
speed  of  movement  on  route 
likely  ambush  sites 
likely  sniper  positions 
windows/rooftops/doorways  with 
best  observation  areas 

traffic  patterns 

neighborhoods  friendly/hostile  to 

US  forces 

projected  availability  of 
cover/concealment 
projected  time  to 
cover/ concealment 
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From  the  GDTA,  then,  we  learn  that  the  company  commander  has  multiple  information 
requirements  facing  him.  He  must  consider  many  different  pieces  of  information  to  understand 
the  complexity  of  the  urban  terrain  and  how  the  buildings,  streets,  and  alley  patterns  impact  on 
his  movement  and  security.  To  compound  the  difficulty,  he  gets  this  data  from  multiple  sources. 
Maps,  written  and  verbal  intelligence  reports,  troop  observation  and  reconnaissance,  local 
civilians  and  his  own  experience  come  together  to  help  him  paint  the  picture  and  confirm  or  deny 
the  impact  the  terrain  may  have  on  his  plan.  The  challenge  for  the  company  commander  is  to 
take  this  information,  integrate  it  together  in  some  fashion  to  effectively  determine  where  to 
position  his  teams,  how  to  methodically  search  the  area,  and  where  best  to  position  his  reserve  to 
react  to  situations  that  arise  during  the  operation. 

In  the  process  he  must  also  continually  consider  the  enemy,  where  enemies  are  likely  to 
be  positioned,  with  what  assets  and  capabilities,  and  how  that  will  impact  the  company 
commander’s  plan  and  mission.  The  company  commander  also  receives  enemy  information 
through  varied  means.  Text,  radio  and  face-to-face  reports  of  past  incidents,  intelligence  reports 
from  higher  headquarters,  observations  and  assessments  of  soldiers  as  they  move  into  and  around 
the  area  must  be  brought  together  to  construct  a  mental  concept  of  likely  enemy  threats.  What 
kinds  of  threats  (RPG  teams,  car  bombs,  snipers,  IEDs  and  booby  traps),  where  the  threat  might 
occur,  and  how  the  enemy  may  react  are  important  areas  where  the  commander  must  apply  his 
experience  and  available  information  to  ensure  he  takes  adequate  measures  to  protect  the  force. 
Information  from  graphical,  textual,  and  verbal  reports  must  be  integrated  to  form  a 
comprehensive  picture  of  the  battlespace  and  the  challenges  the  commander  will  face.  With  little 
training  provided  in  visualization,  such  an  integration  process  relies  heavily  on  the  company 
commander’s  individual  skill  and  experience  and  can,  therefore  be  highly  variable.  Training  that 
targets  the  company  commander’s  ability  to  visualize  and  assess  the  battlefield  during  mission 
planning  will  increase  his  ability  to  effectively  accomplish  his  mission.  The  same  visualization 
skills  that  the  commander  has  learned  and  exercised  during  mission  planning  will  form  the  basis 
for  visualizing  a  rapid  change  during  mission  execution,  even  if  the  visualization  tools  available 
are  during  the  rapid  Replanning  are  only  grease  pencil  and  acetate  and  not  a  C2  system. 

MOUT  Training 

Many  aspects  of  military  leadership  are  described  and  taught  as  both  an  art  and  a  science. 
While  many  important  skills,  such  as  the  MDMP  have  become  well-defined  processes; 
battlefield  visualization  in  support  of  good  decision  making  is  too  often  considered  to  be  only  an 
art.  Like  battlefield  visualization,  art  practice  (e.g.  painting,  sculpting)  is  usually  taught  as  a 
series  of  “how-to”  methods.  But  historically,  as  arts  evolve,  these  how  to  methods  become 
systematized  and  codified  (e.g.  the  development  and  refinement  of  color  theory  in  painting).  A 
central  goal  of  this  STTR  effort  has  been  to  examine  how  a  combination  of  novel  computer 
visualization  techniques,  situation  awareness-centric  task  analysis,  intelligent  tutoring,  and  other 
digital  information  presentation  and  learning  techniques  could  be  used  to  systematize  battlefield 
visualization,  transforming  it  from  an  art  to  a  science.  This  section  discusses  how  such  skills  are 
currently  taught  in  formal  military  courses  (i.e.  “schoolhouse”  training),  an  assessment  of 
battlefield  needs  as  reported  from  several  anecdotal  accounts  from  Operation  Iraqi  Freedom 
(OIF),  and  where  fundamental  gaps  exist  between  what  skills  are  taught  versus  what  skills  are 
required.  This  work  addresses  visualization  and  decision  making  needs  for  company-level 
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commanders  in  MOUT  and  Stability  and  Support  Operations  (SASO)  environments  with  an 
emphasis  on  cordon  and  search  missions. 

Leadership  and  battlefield  skills  for  company-level  MOUT  operations  are  taught  in  the 
Infantry  Captains  Career  Course  at  Ft.  Benning,  via  correspondence  course  (US  Army  Infantry 
School  ACC  071  A1 1),  and  through  similar  courses  taught  for  other  officer  specialties.  While 
there  is  no  specific  lesson  in  Battlefield  Visualization  or  the  use  of  visualization  techniques, 
aspects  of  it  are  taught  throughout  the  course  work  in  the  context  of  other  lessons.  For  example, 
although  there  are  training  modules  for  Military  Operations  other  than  War,  Combat  in  Built-Up 
Areas,  and  Tactical  Doctrine  that  address  theoretical  and  doctrinally  correct  methods  for 
planning,  preparing,  and  executing  various  operations,  none  of  the  training  modules  discusses  the 
practical  techniques  necessary  to  utilize  information  generally  and  graphical  artifacts  in 
particular  to  develop  mental  models  or  schemata  necessary  to  internally  frame  the  battle  situation 
or  to  develop  external  graphical  artifacts  from  existing  multi-source  intelligence  products. 

There  is  also  a  somewhat  disjointed  organization  of  command  doctrine  reflected  in 
current  field  manuals  such  as  FM  90-10  (MOUT,  Department  of  the  Army,  1979)  and  FM  3-06- 
1 1  (Combined  Arms  on  Urban  Terrain,  Department  of  the  Army,  2002a).  This  lack  of 
consistency  carries  over  to  joint  operations  (Joint  Urban  Operations  -  Enabling  Concept  v.86). 

To  help  bridge  the  gap  between  formal  training  and  operational  need,  training  circulars  such  as 
TC  7-98-1,  Stability  and  Support  Operations  Training  Support  Package  (Department  of  the 
Army,  1998),  combine  various  doctrinal  elements  to  address  specific  practical  mission 
requirements  such  as  for  cordon  and  search  operations.  However,  even  these  more  practical 
training  support  materials  do  not  address  visualization  skills  in  a  way  that  can  more  fully  prepare 
new  officers  for  battle  conditions.  TRADOC  PAM  525-70  (Department  of  the  Army,  1995) 
discusses  the  art,  science,  and  integration  of  battlefield  visualization  skills  in  a  general  way.  It 
notes  that  computer  and  digital  technology  are  essential,  but  it  does  not  discuss  how  warfighters 
can  or  should  make  use  of  specific  technologies  to  support  visualization  and  situation  awareness. 
It  also  does  not  discuss  how  such  essential  skills  are  best  connected  with  the  MDMP.  TRADOC 
PAM  525-70  concedes  that  battlefield  visualization  is  still  more  of  an  art  than  a  science. 

The  gap  between  current  formal  training  practices  and  operational  need  has  been 
expressed  in  many  forms  and  at  many  levels,  but  some  of  the  most  useful  anecdotal  accounts  are 
in  the  form  of  “lessons  learned”.  Many  such  accounts  from  OIF,  Bosnia,  and  elsewhere  have 
been  captured,  passed  along,  and  are  used  to  help  structure  in-country  unit  training  of  personnel. 
The  Center  for  Army  Lessons  Learned  (CALL)  serves  as  a  repository  for  many  official  accounts, 
but  websites  such  as  http://companycommand.army.mil  have  been  formed  to  support  the 
exchange  of  information  at  a  much  less  formal  level.  Among  the  official  accounts  from  OIF  is 
the  CALL  Initial  Impressions  report  04-13  (Center  for  Army  Lessons  Learned,  2004)  that 
discusses  specific  gaps  between  doctrine  and  operational  need.  One  focus  area  for  this  report  is 
Information  Operations  and  the  inability  of  many  warfighters  to  fully  understand  information 
operations,  the  collection  process,  or  how  to  integrate  information  at  the  tactical  level.  These 
issues  are  echoed  by  U.S.  Marine  Corps  units  as  well  (US  Marine  Corps,  2003)  and  expound 
further  on  the  inability  to  fully  understand  information  operations  and  how  to  separate  necessary 
from  unnecessary  information.  Developing  better  visualization  skills,  understanding  how  to 
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visualize  what  is  known  and  not  known,  and  how  such  visualizations  feed  into  the  MDMP  would 
help  bridge  this  gap. 

Among  the  unofficial  lessons  learned  accounts  are  two  that  characterize  a  rather  broad 
need  for  better  visualization  skills.  Morgan  (2004)  discusses  lessons  learned  from  the  perspective 
of  an  infantry  company  commander.  Morgan  stresses  the  need  to  maintain  high  levels  of 
situation  awareness  for  all  operations  and  the  ability  to  quickly  recognize  bad  situations  and 
patterns  of  activity.  Although  he  concludes  by  stating  that  most  of  what  is  taught  in  military 
schools  remains  useful,  he  concedes  it  is  not  enough.  To  become  highly  effective,  he 
recommends  that  company  commanders  conduct  extensive  training  and  rehearsals  with  their 
units  using  realistic  terrain  models  and  realistic  missions  (such  as  cordon  and  search,  securing 
convoy  routes,  normal  patrols,  and  civil-military  operations)  and  follow  every  mission  with  an 
after  action  review.  Conducting  such  training  is  the  basis  for  forming  accurate  and  effective 
mental  models;  conducting  after  action  reviews  is  the  basis  for  correcting  and  extending  existing 
mental  models.  It  is  essential  for  effective  commanders  to  be  capable  of  forming  such  mental 
models  themselves,  and  to  be  able  to  teach  those  skills  to  others. 

Olmstead  (2003)  also  presents  an  unofficial  discussion  of  lessons  learned  that  supports 
the  views  of  Morgan.  Among  many  issues,  he  discusses  several  key  gaps  between  formal 
training  and  operational  need  that  could  be  addressed  by  better  visualization  training  tools: 

•  Dismount  drills  for  mechanized  units  -  While  there  exist  good  standard  operating 
procedures  (SOP)s  for  dismounting  under  various  circumstances,  recognizing  the  current 
situation  and  how  it  maps  into  a  specific  action  drill  could  be  improved  by  visualizing  different 
situations  and  understanding  salient  recognition  features. 

•  Communication  and  navigation  in  MOUT  environments  -  Understanding  when  standard 
communications  SOPs  will  not  be  adequate  (e.g.  within-building  or  subterranean  operations), 
understanding  how  to  successfully  visualize  communication  requirements,  and  understanding 
how  and  when  to  augment  SOP  with  ad  hoc  solutions  (e.g.  line  of  sight  communication  chains). 
Additionally  the  ability  to  integrate  and  use  multiple  information  types  (e.g.  map-based  and 
satellite)  is  a  useful  skill  for  unknown  environments. 

•  Convoy  and  team  operations  -  Visualizing  overwatch  locations,  potential  combatant  and 
sniper  locations,  lines  of  sight,  and  other  important  spatial  aspects  of  MOUT  missions  is  critical 
to  effective  planning. 

•  Training  to  ROE  (Rules  of  Engagement)  -  Understanding  how  ROE  will  affect  mission 
conduct  is  an  important  aspect  of  planning  and  preparation.  Visualizing  potentially  hazardous 
situations  and  locations  is  critical  to  understand  when  the  ROE  should  change. 

•  Cultural  training  -  Using  graphical  and  other  means  to  better  understand  the  operational 
environment,  normal  patterns  of  activity  (e.g.  daily  prayer  times),  and  abnormal  conditions  (e.g. 
a  freshly  shaved  beard  can  indicate  an  expectation  to  die)  are  essential  to  situation  recognition 
and  maintaining  high  levels  of  situation  awareness. 

•  How  to  develop  actionable  commanders  critical  information  requirements  (CCIRs)  and 
collection  plans  -  Visualizing  available  information,  information  needs,  information  gaps,  and 
their  connection  to  CCIR  and  battlefield  decisions  are  critical  skills  for  refining  CCIRs  and 
collecting  sufficient  information  to  make  effective  and  timely  decisions. 
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While  Olmstead  does  not  make  a  connection  to  technology  to  address  these  gaps,  it 
seems  clear  that  utilizing  digital  technology  and  developing  courseware  that  specifically 
addresses  these  issues  has  a  strong  potential. 

Review  of  Battlefield  Visualization,  Military  Decision  making,  and  Information  Management 
Literature 

Battlefield  Visualization  is  a  three-step  command  process  whereby  the  commander 
develops  a  clear  understanding  of  the  current  situation,  envisions  a  desired  end  state,  and 
visualizes  the  sequences  of  activity  that  will  move  his  force  from  its  current  situation  to  the 
desired  end  state  (Department  of  the  Army,  2003a).  As  shown  in  Figure  1 1,  the  foundation  of 
battlefield  visualization  is  the  information  loop,  which  consists  of  defining  the  CCIR,  collecting 
data  related  to  those  requirements,  and  transforming  collected  data  into  situational  understanding 
(Department  of  the  Army,  2003a).  This  final  transformation  process  has  traditionally  been 
associated  with  information  fusion.  The  process  of  decomposing  high-level  information  needs 
into  fme-grained  collection  needs  could  be  called  an  “information  fission”  process. 


Figure  11.  Commander's  battlefield  visualization  information  flow. 


To  help  accomplish  the  planning  aspect  of  Battlefield  Visualization,  the  Military 
Decision  making  Process  (MDMP)  is  a  detailed,  regimented  sequence  of  steps  for  developing  an 
operational  plan  to  accomplish  mission  goals  (see  Figure  12).  The  MDMP  starts  with  the  receipt 
of  a  mission  from  higher  command,  steps  through  the  process  of  analyzing  the  mission  and 
developing  a  course  of  action  (CO A)  through  analysis  and  war  gaming,  and  concludes  with  the 
production  and  dissemination  of  an  OPORD  to  the  unit.  At  the  Unit  of  Action  level,  the 
commander  would  be  assisted  in  this  process  by  a  command  staff,  who  commonly  provide  five 
basic  functions  to  commanders  in  support  of  reconnaissance,  security,  offensive,  and  defensive 
operations  (Department  of  Army,  1996): 
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•  Provide  timely  and  accurate  information 

•  Anticipate  requirements  and  prepare  estimates 

•  Determine  courses  of  action  and  make  recommendations 

•  Prepare  plans  and  orders 

•  Supervise  execution  of  decisions 


Figure  12  illustrates  the  steps  involved  in  the  MDMP.  This  process  lays  the  foundation 
for  decision  making  during  execution  of  the  OPORD,  by  specifying  points  in  the  execution  plan 
at  which  decisions  (i.e.,  selections  of  pre-planned  “branches”)  must  be  made. 
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Figure  12.  The  staff  inputs  and  outputs  in  military  decision  making  process. 


As  can  be  seen  in  Figure  12,  one  output  of  the  MDMP  is  the  commander’s  critical 
information  requirements  (CCIR)  (Department  of  the  Army,  1997).  Doctrinally,  CCIR  includes 
Priority  Intelligence  Requirements  (PIR),  Friendly  Force  Information  Requirements  (FFIR),  and 
(somewhat  adjacently)  Essential  Elements  of  Friendly  Information  (EEFI).  This  categorization 
helps  defined  how  information  requests  are  constructed  by  the  commander  and  reported  by 
elements  in  the  field.  The  ability  for  the  commander  to  determine  their  information  needs  and  be 
able  to  define  them  in  a  manner  that  is  actionable  is  critical  to  their  ability  to  monitor  the 
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battlefield  and  to  integrate  battlefield  observations  with  their  situational  knowledge  and  general 
military  knowledge. 

While  TRADOC  (Department  of  the  Army,  1995)  describes  the  need  for  training,  and  the 
requirement  including  training  with  technological  tools,  it  does  not  specify  the  role  of  these  tools 
in  the  training  process.  It  is  clear,  though,  that  there  are  at  least  two  major  roles  of  technological 
tools  in  the  battlefield  visualization  process.  First,  training  is  required  to  support  the  developing 
visual  representations  of  the  current  battlefield  situation  that  support  situation  awareness  and 
pattern  identification.  Regardless  of  the  digital  display  technology  in  theater  and  available  to 
commanders,  technical  tools  can  be  developed  to  support  the  development  of  internal 
representation  construction  expertise.  Second,  technological  tools  can  be  developed  to  support 
and  improve  the  construction  and  analysis  of  visual  representations  based  on  real-time  COP  and 
CCIR  information  feeds.  These  tools  would  support  the  commander  in  theatre  and  would  also 
require  appropriate  training. 

Review  of  Cognitive  Support  for  Visualization  Literature 

External  visual  representations.  In  addition  to  the  work  being  done  in  optimizing  a  data¬ 
set  display  based  on  its  perceptual  features,  there  is  a  long  tradition  of  humans  designing  visual 
representations  to  support  higher-level  communication  and  reasoning.  These  representations  use 
a  range  of  visual  depiction  styles,  showing  concrete  visible  things,  and  graphical  grammars, 
representing  invisible  abstract  concepts.  (Tversky,  2001)  Early  examples  include  drawings, 
paintings,  maps,  and  petroglyphs.  More  contemporary  examples  include  photographs,  television 
and  movie  images,  geographic  information  system  (GIS),  radar  and  meteorological  displays,  as 
well  as  1st  and  3rd  person  computer  game  screens  (see  Figure  13).  Since  depictions  are  based  on 
concrete  objects  or  locations  in  either  the  real  or  a  simulated  worlds  they  tend  to  have  strong 
spatial  alignment  with  that  world  and  can  be  viewed  as  a  structured  viewport  into  that  world. 
This  viewport  does  not  have  to  provide  an  exact  photo-realistic  view,  however.  Research  and 
commercial  systems  have  explored  many  ways  to  add  additional  information  to  a  depiction  by 
manipulating  image  features  including  color  or  luminosity  coding  screen  regions,  smoothing 
road  networks,  or  adding  or  removing  specific  terrain  feature  layers. 
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Figure  13.  Satellite  image  of  the  A1  Salam  Palace  in  Iraq  as  a  representitive  conteporary  depiction. 


Historically,  graphics  evolved  out  of  depictions  to  communicate  concepts  that  were 
difficult  or  impossible  to  directly  depict,  such  as  proper  names,  abstract  entities,  causality, 
quantifications,  and  negations  (Tversky,  2001).  Contemporary  graphic  forms  include  logic 
diagrams  such  as  those  devised  by  Euler  and  Venn  (Shin,  2002)  and  information  graphics  such  as 
bar  charts,  box  plots,  time  lines,  flow  and  organizational  charts  and  network  diagrams  (Harris, 
1999)  The  SASO  flow  diagram  (Department  of  the  Army,  1997)  shown  in  Figure  14  is  a  typical 
example  of  an  information  graphic.  While  graphics  acquired  the  advantage  of  being  able  to 
express  the  abstract  concepts  associated  with  SASO,  they  have  lost  the  transparency  of  realistic 
depictions.  In  addition,  while  information  graphics  usually  have  a  strong  internal  spatial 
orientation,  such  as  vertical  and  horizontal  axis,  this  orientation  only  has  metaphoric  or  analogic 
reference  to  any  external  world.  The  implication  of  this  loss  of  representational  transparency  and 
the  lack  of  external  spatial  referents  is  that,  while  graphics  provide  a  range  of  benefits,  any  use  of 
graphics  generally  requires  specific  training.  Their  meaning,  unlike  that  of  a  depiction,  is  not 
automatically  meaningful  to  the  observer.  Training  and  practice  are  required  to  render  graphics 
intelligible. 
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SIX  3.  NONCOMBATANT  EV  ACUATION  OPERATIONS  (UNCERTAlNi 


Figure  14.  Stability  and  Support  Operations  (SASO)  noncombatant  evacuation  operations 
flow  diagram  is  a  representitive  contemporary  information  graphic. 

When  appropriate,  combining  purely  visual  depictions  and  graphics  can  increase  the 
expressive  power  of  the  representation  by  leveraging  the  strong  spatial  alignment  and  ease  of 
interpretation  of  the  depiction  with  the  abstract  expressive  power  of  the  graphic.  This  approach  is 
often  taken  in  Intelligence  Preparation  of  the  Battlefield,  where  maps  and  satellite  imagery  are 
annotated  with  geometric  areas  of  interest  (See  Figure  15;  GlobalSecurity.org,  1994)  and  military 
Course  of  Action  sketches  (see  Figure  16;  Department  of  the  Army,  1997),  where  abstract  time- 
based  operational  graphics  are  annotated  on  local  terrain  maps. 
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Figure  15.  Satellite  map  depiction  of  Baghdad  using  graphics  and  text 
annotations  to  define  regions  of  interest. 
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Figure  16.  Course  of  action  sketch  overlays  a  map  depiction  with  military  operational 
graphics  including  entity  and  region  identifications  and  temporal  flow  markers. 

Benefits  of  cognitive  representations.  Researchers  in  cognitive  psychology, 
mathematics,  and  a  range  of  other  disciplines  have  identified  a  number  of  basic  principles  that 
underlie  the  construction  of  external  visual  representations  that  support  improved  human 
reasoning  or  task  performance.  These  principles  generally  focus  on  two  main  points.  The  first 
point  is  that  representations  can  improve  computational  efficiency  (Larkin  &  Simon,  1987)  by 
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making  features  more  easily  recognizable,  making  information  more  explicit,  searchable,  or 
interpretable.  The  second  point  is  that  representations  can  improve  domain  learning 
characteristics  (Cheng,  1999)  by  providing  the  learner  with  an  external  representation  that 
encodes  all  relevant  features  of  a  problem  space  and  helping  to  promote  the  integration  of  those 
features.  Table  2,  below,  is  adapted  from  and  extends  (Woods,  1994)  and  presents  a  set  of  core 
performance  and  learning  issues  that  can  be  supported  by  carefully  constructed  representations. 

Table  2.  Value  of  external  representation  in  supporting  human  task  performance  and 
learning 


Task  Performance  or  Learning  Issue 

Value  of  Representation 

1.  Problem  Structuring 

A  good  external  representation  is  constructed  in  a 
form  that  allows  different  strategies  to  be  used.  For 
example,  maps  are  structured  to  allow  spatial 
calculations  such  as  relative  distance  and  areas  of 
effect  to  be  made  easily. 

2 .  Overload/  workload 

A  good  external  representation  reduces  or 
attenuates  cognitive  effort  by  allowing  processing 
that  may  be  parallel,  perceptual,  or  reentrant.  In 
addition,  a  good  representation  can  reduce 
demands  on  memory  or  dependencies  on  attention. 

3. Control  of  Attention 

A  good  external  representation  can  draw  attention 
to  specific  regions  of  the  representation  to  support 
important  data  properties.  A  good  external 
representation  can  also  hold  a  viewers  attention, 
allowing  them  to  conduct  more  in  depth  analysis. 

4. Secondary  Tasks 

A  poor  representation  can  create  unnecessary 
secondary  tasks,  such  conversions  between  scales 
and  correlating  with  unrepresented  data  ranges.  A 
good  representation  can  create  or  allow  beneficial 
secondary  tasks  that  might  be  skipped,  such  as 
consistency  and  cross-checking  results. 

5. Effort 

A  good  representation  can  reduce  effort  for  a 
specific  task-context. 

6. Communication  &  Coordination 

A  good  representation  allows  others  access  to  the 
problem  solving  process  and  intermediary  results. 

7.Guided  Interpretation 

A  good  representation  supports  the  range  of  valid 
interpretations  that  can  be  made  of  a  set  of 
information  and  does  not  support  improper 
interpretations 
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Internal  Representations.  The  external  visual  representations  described  above  are 
directly  related  to  human  internal  representations,  also  known  as  mental  models  or  schema. 

There  is  a  core  relationship  between  human  internal  representations  and  external  representations 
of  problem  areas.  This  relationship  has  three  principle  components  (Norman,  1983): 

•  Belief  System:  The  belief  system  that  forms  the  schema  has  been  acquired  through 
observation,  instruction,  or  inference. 

•  Observability:  There  is  a  correspondence  between  parameters  and  states  in  the  humans’ 
schema  and  the  parameters  and  states  that  the  human  can  observe  in  the  external  world. 

•  Predictive  Power:  The  purpose  of  the  schema  is  to  enable  limited  prediction  of  future 
states  and  parameters.  This  means  that  the  model  learned  by  the  human,  to  be  useful,  must  have 
the  ability  to  generate  useful  predictions.  By  implication,  the  external  belief  system  the  human  is 
engaged  with  must  be  structured  to  support  the  same  predictions  and  to  enable  the  human  to 
make  those  predictions. 

For  example,  the  COA  graphic  (see  Figure  16,  on  page  30)  layers  a  complex  information 
graphic  on  top  of  simple,  stylized,  map  depiction.  The  authors  and  readers  of  this  diagram  need 
specific  training  in  understanding  the  syntactic  content  of  the  individual  pictograms  and  the 
semantic  meaning  of  their  relative  locations  on  the  base  map.  The  graphic  presents  a  very 
specific  class  of  tactical  mission  pictograms  that  mark  the  location  of  concrete  objects, 
including  friendly  units  and  town  locations,  abstract  geographic  areas,  such  as  objective  areas, 
and  abstract  process  flow  indicators,  such  as  advance  and  reconnaissance  arrows  and  lines  of 
advance.  This  graphic  both  reflects  and  informs  how  the  military  understands  courses  of  action 
and  the  critical  behaviors  of  Army  units  and  renders  the  graphic  authors  understanding  of  the 
current  world  and  possible  future  world  explicit  and  observable.  The  graphic  has  a  strong  spatial 
and  temporal  orientation,  articulating  the  behavior  of  entities  over  time  and  space.  The  graphic 
only  reflects  intent,  however,  and  not  a  scripted  reality.  COA  diagrams  have  only  a  limited 
predictive  power,  but  a  strong  interpretive  and  communicative  power. 

One  critical  point  is  that  schemas  are  not  simply  internalized  versions  of  external 
representations.  Schemas  are  flexible  and  active,  automatically  filling  in  missing  information, 
generalizing  from  the  past.  They  are  continually  modifying  and  adapting  interpretive  states 
(Norman,  1986b)  that  guide  how  we  think  understand  our  environment.  These  correspondences, 
then,  apply  broadly  to  the  entire  environment  and  not  just  to  visual  representations.  In  the  COA 
example,  a  trained  reader  can  identify  detail  not  available  in  the  external  representation  and 
apply  general  knowledge  to  fill  in  those  gaps. 

The  impact  on  visual  representations,  though,  is  clear:  external  representations  must 
support  the  construction  of  mental  models  with  appropriate  expressiveness  and  predictive  power. 
A  COA  map  that  expresses  historic  meteorological  data  but  not  planned  actions  would  not  be 
appropriate.  In  addition,  external  representations  must  provide  indicators  of  current  parameters 
and  states  in  a  manner  that  corresponds  with  that  model.  Finally,  the  external  representation  must 
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Standard  military  terminology  refers  to  these  pictograms  as  “graphics.”  We  have  used  the  more 
specific  term  “pictogram”  here  to  avoid  confusion  with  the  broader  class  of  information  graphics. 
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support  the  interactive  construction,  modification,  and  interpretation  to  support  the  performance 
and  learning  issues  described  above. 


There  is  a  large  challenge,  then,  in  designing  external  visual  representations  that  have  the 
right  mix  of  concrete  depiction  and  abstract  graphics  laid  out  in  a  manner  that  supports  the  right 
class  of  calculations,  search  and  inferences  and  populated  with  data  that  properly  reflects  the 
external  and  internal  worlds.  These  are  significant  challenges,  particularly  when  we  consider  the 
task  dependent  nature  of  representations.  Table  2  (see  page )  listed  a  number  ways  that  external 
representations  impact  task  and  learning  performance.  The  implication  of  this  list  is  that  task 
specific  representations  better  mesh  and  support  schemas  and  improve  task  and  learning 
performance.  The  challenge,  then,  is  to  ensure  that  a  human  engaged  in  a  performance  or 
learning  task  either  has  available  or  can  construct  an  appropriate  external  representation. 

Basic  Visualization  Skills.  In  pursuing  this  research  we  have  developed  a  set  of  basic 
skills  that  are  required  to  support  the  construction,  manipulation  and  usage  of  external 
representations  (see  Table  3)  and  their  positive  usage  in  internal  model  formation  and  decision 
making  (see  Table  4).  These  skills,  we  believe,  are  both  applicable  to  the  user  of  a  visual 
representation  and  trainable  in  a  general  way  that  will  support  transfer  of  skill  across  a  range  of 
tasks  and  technologies.  Training  these  skills  requires  a  focus  on  the  information  and 
computation  needs  of  the  task  in  question  and  understanding  the  nature  of  visual  grammar  such 
that  representations  can  be  constructed  that  meet  these  requirements. 


Table  3.  Skills  for  External  Representation  Manipulation 


1 .  Identify  their  current  information  needs 

2.  Select  an  appropriate  visual  artifact 

3.  Interpret  the  artifact,  determining: 

•  What  classes  of  information  it  contains 

•  What  classes  of  inference  it  supports 

•  What  classes  of  information  and  inference  are  not  supported 

4.  Manipulate  the  artifact  to  put  it  in  a  state  that  enables  the  desired 

information  to  be  extracted  and  inferences  made _ 

Table  4.  Skills  for  Internal  Schema  Construction 


1 .  Extract  relevant  information  from  the  prepared  visual  artifact 

2.  Convert  the  information  into  a  form  that  fits  their  existing 
mental  schema,  including: 

•  Transforming  information  from  one  representation  or  orientation 
into  another 

•  Aggregating  discrete  information  chunks 

•  Refining  the  existing  schema  (as  necessary) 

3.  Draw  inferences  from  new  knowledge  to  support  decision 

making  process _ 
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Visual  language  theory  suggests  that  humans  process  diagrammatic  graphics  in  a  manner 
similar  to  natural  language  and  that  visual  languages,  including  all  the  graphics  examples  above, 
have  similar  structure  including  a  visual  syntax,  semantics,  and  pragmatics  (Narayanan  & 
Fliibscher,  1998).  Syntax  describes  the  grammatical  form  of  the  language:  what  elements  it 
contains  and  how  those  elements  are  structurally  organized. 

Semantics  describes  how  statements  made  with  the  language  are  interpreted,  and  what 
interpretations  are  valid.  Pragmatics  describes  how  we  can  use  the  meaning  of  the  statements  to 
achieve  specific  goals. 

For  an  example,  consider  the  MOUT  Intelligence  Preparation  of  the  Battlefield  incident 
overlay  map  below  (Medby  &  Glenn,  2002,  see  Figure  17).  Proper  interpretation  of  this  map 
requires  a  thorough  knowledge  of  the  syntax,  semantics,  and  pragmatics  that  were  assumed  in  its 
construction.  The  map  syntax  defines: 

1.  The  type  of  marks  used  to  encode  roadways  of  different  sizes,  i.e.  lines  of  different  weights 
where  an  increase  in  weight  corresponds  to  an  increase  in  road  carrying  capacity  and/or  lane  size. 

2.  The  types  of  marks  used  to  encode  roadway  names  and  roadway  types,  i.e.  full  text  of  a 
specific  size  to  note  urban  streets  and  larger  text  in  a  circle  to  note  a  highway  system. 

3.  The  types  of  marks  used  to  encode  incident  data,  i.e.  •  for  ambush,  ★  for  bombing,  □  for 
murder,  and  O  for  kidnap. 

The  incident  overlay  map  semantics  shows  that  there  are  twelve  individual  incidents  in 
this  region  in  the  time  frame  recorded  and  that  there  is  one  cluster  of  bombings,  one  cluster  of 
murders,  two  clusters  (or  one  large  cluster)  of  ambushes  and  kidnaps.  The  map  pragmatics 
supports  historic  functions,  including  the  recording  of  incident  data  and  analytic  functions, 
including  the  identification  of  incident  locations  and  incident  patterns. 

It  is  equally  important  to  understand  what  the  visual  language  used  in  this  map  does  not 
support.  While  the  map  implies  that  it  records  incidents  over  a  limited  period  of  time,  the  map 
syntax  provides  any  marks  that  would  date  the  individual  incidents  in  the  map  or  the  duration  of 
time  covered  by  the  map  itself.  From  a  semantics  perspective  then,  it  is  impossible  to  extract 
temporal  patterns,  such  as  inferring  the  direction  that  the  bomber  might  be  moving  through  the 
city.  From  a  pragmatics  perspective  then,  this  map  is  inappropriate  for  identifying  temporal 
patterns. 
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Figure  17.  Interpreting  this  MOUT  Incident  Overlay  Map  requires  understanding  very  specific  syntax, 
semantics,  and  pragmatics 

Even  more  critically,  this  map  only  provides  syntactic  markings  that  show  positively 
identified  incidents.  The  semantic  implication  is  that  any  location  without  a  marking  has  had  no 
relevant  incident  within  the  time  period  of  the  map.  This  may  be  a  false  inference,  but  one 
common  to  map  based  representations.  There  are  a  number  of  reasons  why  relevant  information 
may  be  unreported,  including  lack  of  observer  or  lack  of  observer  motivation  or  capability  to 
report  the  incident.  This  specific  issue  can  have  great  impact  on  a  military  map,  where  enemy 
units  are  marked  based  on  sensor  or  reconnaissance  observations.  The  fact  that  an  area  on  the 
map  has  no  enemy  units  marked  may  be  simply  that  the  area  in  question  is  outside  of  sensor 
range  or  that  no  reconnaissance  team  has  coverage  in  that  area.  Unfortunately  typical  military 
maps,  such  as  the  one  in  Figure  17,  have  this  breakdown  between  actual  map  syntax  and  the 
apparent  semantics.  This  can  be  contrasted  to  the  National  Oceanic  and  Atmospheric 
Administration’s  National  Weather  Service  (NOAA  2005)  radar  image  below  (see  Figure  18). 
The  syntax  of  this  map  includes  marks  for  meteorological  radar  sightings  as  well  as  marks  that 
describe  the  limits  of  radar  coverage. 

While  the  examples  described  here  are  drawn  from  computer  rendered  mapping  and 
display  systems,  the  issues  at  hand  are  not  inherently  technological.  The  same  issue  could  arise 
on  a  map  annotated  with  a  grease  pencil  in  an  in- theatre  command  vehicle.  Instead,  it  highlights 
a  common  visual  language  syntactic  and  semantic  breakdown  that  can  form  between  external 
representations  and  internal  schemas.  It  also  points  to  the  potential  benefits  that  training  can 
provide.  Once  made  aware  of  this  representation  problem  and  the  methods  to  detect  it  in  other 
representations,  we  would  expect  that  the  knowledge  would  transfer  to  other  similar 
representations. 
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Figure  18.  A  National  Weather  Service  radar  image  with  visual  syntax  describing  marks  for  radar  sightings 
and  radar  coverage 

Review  of  Educational  Technology  Literature 

Learner-centered  design  vs.  user-centered  design.  The  MAVEN-SA  project’s  desire  to 
develop  a  system  to  train  company  commanders  in  battlefield  visualization  skills  requires  a 
strong  focus  on  a  learner-centered  design  approach.  A  traditional  user-centered  design  (UCD) 
approach  is  used  when  designing  software  to  assist  users  in  their  work  (Norman,  1986a).  The 
target  audience  (the  user)  already  understands  the  basics  of  the  work  practice  but  needs  a  tool  to 
help  them  complete  their  work  more  easily  and  effectively.  The  user  is  not  necessarily  trying  to 
learn  about  their  work  through  use  of  the  tool.  In  fact,  UCD  tool  designers  often  work  explicitly 
to  remove  the  necessity  for  users  to  learn  anything,  believing  that  users  wish  to  minimize 
learning  requirements  (Carroll  and  Rossen,  1987).  The  design  process  must  address  the 
conceptual  gulfs  between  the  user  and  the  tool  (Quintana,  Krajcik,  &  Soloway,  2003;  Quintana, 
Krajcik,  &  Soloway,  2001).  More  specifically,  the  execution  of  actions  on  the  tool  must  be 
straightforward  and  consistent  with  the  user’s  goals,  and  the  evaluation  of  the  state  of  the  tool 
must  be  understandable.  UCD  designers  often  conceptualizing  tool  use  as  a  problem  solving 
task  where  the  problem  solving  is  rendered  as  simple  and  well  supported  as  possible  (Card, 
Moran  and  Newell,  1983).  These  usability  issues  tend  to  be  the  primary  focus  of  UCD. 
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On  the  other  hand,  a  learner-centered  design  (LCD)  approach  is  used  when  designing 
software  to  not  only  assist  users  in  performing  some  unfamiliar  task,  but  also  to  help  them  learn 
the  knowledge  and  skills  involved.  The  target  audience  in  this  case  (the  learner)  has  unique  needs 
(Soloway,  Guzdial,  &  Hay,  1994).  They  need  support  to  engage  in  the  new  processes,  to  make 
sense  of  the  new  content,  and  to  articulate  their  developing  understanding.  Although  the  design 
process  must  address  issues  of  usability,  it  must  also  address  the  conceptual  gulf  between  the 
novice  and  the  expert  (Quintana  et  al,  2003;  Quintana  et  al,  2001).  An  analysis  of  the  learning 
goals  and  tasks  and  an  understanding  of  the  learner’s  knowledge  state  reveal  what  will  be 
difficult  for  the  learner.  Good  learner-centered  tools  provide  needed  guidance  and  support  so 
that  the  learner  can  engage  in  the  new,  unfamiliar  task  while  gaining  expertise. 

Educational  software  informed  by  theories  of  learning.  Over  the  years,  many  types  of 
software  have  been  developed  to  support  student  learning.  The  various  types  of  educational 
software  can  be  easily  categorized  according  to  their  underlying  theories  of  learning.  The 
following  summary  of  educational  software  approaches  shows  a  chronology  of  learning  theories 
that  have  direct  implications  for  the  kinds  of  knowledge  and  skills  that  can  be  taught  (Quintana  et 
al,  2003).  Because  the  visualization  skills  that  the  MAVEN-SA  project  is  aiming  to  teach  are 
complex  cognitive  processes,  certain  approaches  are  more  applicable  than  others. 

Behaviorist  approach  to  educational  technology:  Teaching  machines.  B.  F.  Skinner 
(1958)  viewed  learning  as  a  “programming”  process  whereby  a  person’s  externally  visible 
behavior  could  be  shaped  through  a  conditioning  system  of  rewards  and  punishments.  This 
behaviorist  school  of  thought  led  to  the  development  of  a  class  of  “teaching  machines”  often 
referred  to  as  computer-based  training  (CBT)  or  computer-aided  instruction  (CAI).  Immediate 
feedback  was  perhaps  the  most  prominent  feature  of  such  software  tools.  Learner’s  responses 
are  compared  to  pre-programmed  answers  and  appropriate  positive  or  negative  feedback  is 
given.  In  addition,  a  pre-programmed  rubric  is  used  to  determine  whether  the  learner  has 
successfully  learned  enough  (i.e.,  gotten  enough  correct  responses)  to  proceed  to  the  next  task. 
According  to  Beck,  Stem,  &  Haugsjaa  (1996)  here  is  no  mechanism  to  evaluate  the  learner’s 
knowledge  or  needs  beyond  this  shallow  level. 

The  behaviorist  approach  is  only  useful  in  cases  where  the  tasks  are  narrowly  defined  and 
have  clear  answers.  Examples  of  behaviorist  software  include  many  educational  games  and 
“flash  card”  type  software.  Because  educational  reform  efforts  are  calling  for  an  emphasis  on 
higher-level  thinking  skills  and  more  open-ended  problem-solving  tasks,  behaviorist  software  is 
no  longer  popular  in  current  educational  software  research  and  practices  (Anderson,  Reader,  & 
Simon,  1998). 

Information  processing  approach  to  educational  technology:  Intelligent  tutoring 
systems.  In  contrast  to  the  behaviorist  approach  that  considers  only  observable  behavior,  the 
information  processing  approach  considers  models  of  cognition  that  are  based  on  the  idea  that 
“certain  aspects  of  human  cognition  involve  knowledge  that  is  represented  symbolically” 
(Anderson,  1983).  Rules  can  be  applied  to  the  representations  to  manipulate  the  knowledge, 
generate  new  knowledge,  or  make  inferences.  Human  cognition  can  then  be  represented 
symbolically.  Programs  using  these  representations  and  rules  can  then  be  written  as  models  that 
simulate  human  problem-solving  behavior. 
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The  defining  feature  of  intelligent  tutoring  systems  (ITS)  is  that  they  carefully  oversee  a 
learner’s  work  to  provide  needed  guidance.  ITSs  incorporate  a  rule-based  expert  model  of  the 
target  skill  that  is  used  to  monitor  and  guide  novice  learners  as  they  engage  in  the  new  activity. 
The  intent  of  an  ITS  is  to  model  the  actions  and  interventions  of  a  human  tutor  which  is  the  most 
effective  means  of  instruction  (Bloom,  1984).  The  ITS  uses  information  about  the  task  and  the 
current  state  of  the  learner’s  knowledge  of  that  task  to  make  instructional  interventions  (Corbett, 
Koedinger,  Hadely,  2001).  The  pedagogical  nature  of  these  instructional  interventions  is  quite 
limited  in  nature.  The  model-tracing  approach  of  many  ITSs  forces  the  learner  to  proceed  in 
steps  of  a  specified  grain  size  (corresponding  to  the  underlying  rules)  that  constrains  the 
progression  of  the  learner’s  actions.  If  the  learner  makes  a  recognizable  error  (which  has  been 
pre-programmed  as  a  buggy  rule  in  the  system),  an  error  message  is  presented  to  explain  why  the 
action  is  in  error.  If  the  learner  asks  for  help,  a  message  is  presented  to  guide  the  student  toward 
the  correct  solution  path.  These  error  and  help  messages  are  not  general,  but  instead  are  very 
context  specific  as  they  are  generated  based  on  a  matching  of  the  learner’s  solution  with  the 
underlying  model  of  expert  solution  (Anderson,  Corbett,  Koedinger,  &  Pelletier,  1995). 

The  information  processing  approach  is  useful  in  cases  where  the  representation  of  the 
domain  knowledge  has  a  significant  rule  component.  That  is,  the  domain  must  not  be  primarily 
declarative  knowledge  with  limited  inferential  reasoning  (Anderson  et  al,  1 995).  This  may  seem 
constraining,  but  over  the  years  there  have  been  a  wide  variety  of  ITSs  developed  in  a  number  of 
domains  including  geometry,  medical  diagnoses,  mammography  interpretation,  physics  problem 
solving,  computer  programming  and  algebra  proofs. 

Social  constructivist  approach  to  educational  technology:  Learning  environments.  The  social 
constructivist  approach  is  the  basis  for  many  current  educational  approaches  in  science  (Singer, 
Marx,  Krajcik,  &  Clay  Chambers,  2000).  The  underlying  theory  about  how  people  learn  has  two 
important  pieces.  First,  the  social  piece  asserts  that  “knowledge  is. . .  in  part  a  product  of  the 
activity,  context,  and  culture  in  which  it  is  developed  and  used”  (Brown,  Collins,  &  Duguid, 
1989).  This  contextualized  view  of  knowledge  implies  that  learners  must  participate  in  social 
context  that  reflects  the  culture  of  the  practice.  Second,  the  constructivist  piece  asserts  that 
learners  must  be  actively  engaged  to  make  cognitive  connections  between  their  existing 
knowledge  and  the  knowledge  they  are  learning  (Papert,  1993;  Piaget,  1954). 

Because  constructivism  is  a  “theory  of  knowing”  and  not  a  “theory  of  teaching,”  there  is 
no  one  specific  constructivist  approach  (Bransford,  Brown,  &  Cocking,  2000).  As  a  result,  the 
products  of  the  social  constructivist  approach  are  complex  learning  environments  rather  than 
specific  types  of  instructional  systems.  Learning  environments  can  include  multiple  components 
that  work  together  to  support  the  learner  as  they  mindfully  engage  in  and  learn  a  new  practice. 
Wilson  (1996)  defines  such  a  learning  environment  as:  “a  place  where  learners  may  work 
together  and  support  each  other  as  they  use  a  variety  of  tools  and  information  resources  in  their 
guided  pursuit  of  learning  goals  and  problem-solving  activities.” 

Designers  of  learning  environments  are  guided  by  a  set  of  seven  important  pedagogical 
goals  (Honebein,  1996). 

1 .  Learners  must  be  given  some  autonomy  in  the  learning  process  so  that  they  are  actively 
engaged  in  the  knowledge  construction  process. 
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2.  Learners  must  experience  multiple  ways  to  think  about  and  solve  problems  to  enrich  their 
understanding. 

3.  The  learning  must  be  situated  in  a  realistic  and  relevant  context  to  increase  the  likelihood  of 
transfer  from  the  learning  context  to  actual  practice. 

4.  The  learner  must  be  given  some  ownership  in  the  knowledge  construction  process  so  that  the 
role  of  the  instructor  becomes  supportive  rather  than  primary. 

5.  Collaboration  must  be  encouraged  so  that  the  social  interactions  and  roles  of  the  practice  can 
be  realized. 

6.  Multiple  modes  of  representation  must  be  employed  to  demonstrate  different  perspectives 
and  enrich  the  learner’s  knowledge. 

7.  Metacognitive  processes  must  be  encouraged  so  that  the  learner  can  inspect  and  reflect  upon 
his/her  own  thinking. 

Depending  on  the  context  of  the  targeted  practice,  these  goals  may  be  instantiated  in  a 
variety  of  ways,  leading  to  learning  environments  that  look  very  different  on  the  surface.  A 
learning  environment  may  be  comprised  of  multiple  components  (e.g.,  teacher,  curriculum,  and 
ITS)  that  work  together  to  support  and  guide  the  learner.  Although  it  is  possible  to  construct  a 
learning  environment  that  does  not  incorporate  technology,  it  is  more  practical  to  include 
technology  so  that  the  driving  goal  of  one-to-one  instruction  can  be  more  readily  realized. 

The  social  constructivist  approach  allows  one  to  tackle  more  open-ended  and  exploratory 
tasks  than  either  the  behaviorist  or  information  processing  approaches.  The  very  nature  of  these 
open-ended  and  exploratory  tasks,  however,  requires  the  development  of  more  complex 
cognitive  processes  on  the  part  of  the  learner.  Considering  the  potentially  great  gap  between  the 
novice  learner  and  the  targeted  practice,  significant  structure  and  support  is  needed  for  the  novice 
learner  to  effectively  engage  in  the  new  practices. 

Scaffolding.  In  all  of  the  approaches  to  educational  technology  described  above,  learners 
are  faced  with  novel  tasks  that  are  beyond  their  abilities.  The  intent  is  that  with  support  they  will 
learn  to  do  them.  This  basic  strategy  is  theoretically  based  upon  Vygotsky’s  (1978)  zone  of 
proximal  development  (ZPD)  concept.  ZPD  is  defined  as  the  zone  of  activity  in  which  a  person 
can  produce  with  assistance  what  they  cannot  produce  alone.  Wood,  Bruner,  and  Ross  (1976) 
first  introduced  the  idea  of  scaffolding  as  a  process  to  take  advantage  of  the  ZPD.  They  define 
scaffolding  as  a  process  where  assistance  is  provided  to  enable  learners  to  successfully  perform 
tasks  that  would  be  otherwise  be  too  difficult.  Pea  (2004)  points  out  that  a  fundamental  aspect  of 
the  scaffolding  process  as  an  instructional  strategy  is  fading.  If  the  supports  provided  to  the 
learner  do  not  fade  over  time,  the  learner  may  become  reliant  on  them  and  never  achieve 
autonomous  performance  that  is  the  goal  in  an  instructional  setting.  Situations  in  which  supports 
remain  in  place  and  continue  to  assist  performance  must  be  considered  to  be  distributed 
intelligence  (scaffolds-for-performance)  not  independent  performance  (scaffolds-with-fading) 
(Pea,  2004). 

Although  various  researchers  have  proposed  guidelines  and  strategies  for  implementing 
scaffolding,  there  exists  no  agreed-upon  theory  of  pedagogical  support  nor  mechanisms  to 
describe  successful  scaffolding  approaches  (Quintana  et  al,  2004).  Guzdial  (1994)  first  proposed 
three  ways  software  could  provide  scaffolding  to  provide  needed  structure  for  difficult  tasks: 
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communicating  process  to  learners,  coaching  learners  with  hints,  and  prompting  for  articulation 
and  reflection.  Pea  (2004)  has  recently  proposed  three  scaffolding  strategies:  constraining  tasks 
to  reduce  the  degrees  of  freedom  and  increase  chances  for  successful  performance,  focusing 
learner  attention  by  highlighting  relevant  task  features,  and  modeling  advanced  solutions. 
Although  these  guidelines  for  designing  scaffolds  are  all  theoretically  grounded,  there  are  no 
clear  suggestions  for  how  to  implement  them  in  the  design  of  educational  technology. 

In  an  attempt  to  guide  actual  design,  Quintana  et  al.  (2004)  have  proposed  a  scaffolding 
design  framework  that  builds  on  current  proposals  of  general  scaffolding  principles  (Linn,  Davis, 
&  Eylon,  2004;  CILT,  2004).  This  framework  is  theoretically  grounded  in:  (a)  cognitive 
apprenticeship  (Collins,  Brown,  &  Newman,  1989)  which  specifies  how  performance  of  complex 
tasks  can  be  distributed  with  others  providing  assistance,  (b)  cognitive  models  of  learning  by 
doing  (Anderson,  1983;  VanLehn,  1989)  which  specify  expertise  and  learner  difficulties,  and  (c) 
social  constructivism  (discussed  previously).  The  framework  provided  by  Quintana  et  al  (2004) 
organizes  scaffolding  guidelines  around  three  central  components  of  scientific  inquiry: 
sensemaking,  process  management,  and  articulation  and  reflection.  The  guidelines  describe  the 
kinds  of  support  learners  need  to  perform  each  of  those  inquiry  activities.  Explicit  scaffolding 
strategies  are  then  provided  for  each  guideline  which  provide  concrete  ways  these  guidelines 
could  be  realized  in  software  design. 

There  are  still  no  specific  prescriptions  for  how  to  implement  scaffolding.  Specific 
design  decisions  must  be  based  on  context  and  an  analysis  of  the  learner’s  obstacles.  The 
guidelines  and  strategies  provided  by  research  can  guide  these  decisions.  Scaffolding  can  be 
incorporated  in  any  kind  of  educational  technology.  The  strategies  available  for  use,  however, 
are  limited  by  the  theory  of  learning  which  drives  the  instructional  design.  A  richer  array  of 
scaffolding  strategies  is  available  for  use  in  a  constructivist  learning-environment  than  in  an 
intelligent  tutoring  system. 

Evaluation.  The  final  phase  of  any  development  cycle  involves  evaluation.  In  UCD,  the 
evaluation  is  mostly  focused  on  the  usability  of  the  tool.  In  LCD  however,  the  evaluation  must 
assess  how  well  the  educational  product  (e.g.,  ITS  or  learning  environment)  supports  the  learner 
as  they  perform  and  learn  the  target  practice.  Two  very  different  evaluations  are  needed  to 
determine  the  efficacy  of  the  educational  product  (Salomon,  Perkins,  &  Globerson,  1991). 

An  “effects  of’  evaluation  provides  global  and  summative  information  about  the 
educational  utility  of  the  product  as  a  whole.  Such  evaluations  focus  on  changes  in  learners’ 
understanding  after  they  have  used  the  product.  Traditional  methods  for  doing  “effects  of’ 
evaluations  include  pre  and  post  testing  and  controlled  studies,  where  comparisons  are  made 
between  learning  with  and  without  the  product.  Such  evaluation  methods  allow  one  to  make 
general  conclusions  about  the  effectiveness  of  the  educational  product. 

An  “effects  with”  evaluation  provides  local  and  formative  information  about  how 
learners  interact  with  various  features  of  the  educational  product.  Such  evaluations  provide 
much  richer  information  than  “effects  of’  evaluations  and  therefore  help  create  a  more  detailed 
profile  of  product  use.  There  are  a  variety  of  techniques  used  to  gather  this  type  of  “effects  of’ 
information  but  most  have  an  observational  aspect.  Developers  may  ask  learners  to  think  aloud 
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as  they  interact  with  the  product  or  view  video  of  learners  in  an  attempt  to  understand  the 
conditions  leading  to  impasses  and  how  any  provided  assistance  was  used  by  the  learner. 
Evaluation  methods  can  also  focus  more  directly  on  specific  supports  provided  by  the  system  to 
assess  their  usability  and  utility  to  the  learner. 

Some  recommendations  for  MA  VEN-SA  training  project.  In  light  of  this  review  of 
instructional  design,  theories  of  learning,  educational  technology,  scaffolding  techniques,  and 
evaluation,  we  propose  the  following  directions  for  phase  II  of  this  project. 

•  Application  of  a  learner-centered  design  approach.  The  currently  implicit  expertise  that 
is  built  in  to  the  current  battlefield  visualization  tool  must  be  made  explicit  and  accessible  to  the 
learner.  A  careful  task  analysis  will  help  to  determine  parts  of  the  target  task  which  are  likely  to 
cause  the  learner  difficulty,  and  thus  require  support  to  meet  learner  needs. 

•  Design  of  a  scaffold-rich  learning  environment.  A  social  constructivist  approach  is  the 
most  appropriate  for  teaching  the  targeted  complex  battlefield  visualization  practices.  By 
embedding  the  existing  tool  in  a  system  of  supports,  learners  will  participate  in  a  context  that 
reflects  the  language,  tools,  culture  and  representations  of  the  practice. 

•  Performing  “effects  of’  and  “effects  with”  evaluations.  Although  the  client  of  this 
training  development  project  will  be  most  interested  in  the  “effects  of’  evaluation,  the  “effects 
with”  evaluation  will  be  invaluable  during  the  design  process  and  for  research  purposes. 

Review  of  Military  Applications  of  Digital  Training  Literature 

The  Defense  Science  Board  report  on  Training  Superiority  and  Training  Surprise  (DSB, 
2001)  and  the  subsequent  report  on  Training  for  Future  Conflict  (DSB,  2003)  both  identify 
training  as  an  area  with  great  potential  if  implemented  correctly,  and  a  great  risk  if  not  addressed 
more  pervasively.  As  new  technologies  are  developed  and  fielded  as  part  of  DoD-wide 
transformation,  this  potential  and  risk  becomes  even  more  pronounced.  Scaffolding  can  be  a  key 
enabler  for  implementing  a  broad  class  of  military  training  technologies  that  is  well-supported  by 
the  military  training  literature.  Schaab  and  Moses  (2001)  note  that  individualized  training  can  be 
up  to  five  times  more  effective  than  group  training.  By  developing  training  tools  and  materials 
around  core,  measurable  goals,  the  amount  of  support  needed  can  be  customized  to  individual 
learners.  They  also  call  for  the  type  of  constructivist  approach  to  instruction  for  which 
scaffolding  was  specifically  designed.  By  putting  the  learner  closer  to  the  context  in  which  the 
material  will  be  used  there  is  a  much  higher  potential  for  transfer  of  learning.  The  scaffolding 
approach  is  also  fundamental  for  automatically  controlling  training  conditions  (Lickteig,  2000) 
and  can  be  implemented  in  a  variety  of  ways  to  enable  individual  and  team  training.  As  DoD 
transformation  continues,  new  forms  of  training  that  include  concepts  such  as  scaffolding  will  be 
essential  to  training  all  necessary  aspects  of  technology  use,  beyond  traditional  military  concepts 
to  include  both  the  use  of  new  technology  and  how  to  apply  it  in  military  operations  (e.g.  Lynch, 
2001). 

Visual  Display  Construction  Support  for  Visualization  Training 

Visualization  is  the  conversion  of  collections  of  strings  and  numbers  (or  datasets,  as  they 
are  often  called)  into  images  that  allow  viewers  to  perform  visual  exploration  and  analysis  and 
the  viewer’s  interpretation  of  these  images.  Visualization  begins  with  the  construction  of  a  data- 
feature  mapping  that  converts  the  raw  data  into  images  that  are  presented  to  the  viewer.  An 
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effective  visualization  chooses  a  data  mapping  feature  to  support  the  exploration  and  analysis 
tasks  the  viewer  wants  to  perform.  Multidimensional  techniques  must  address  both  the  size  and 
the  dimensionality  of  a  dataset.  The  challenge  is  to  design  visualizations  that  represent  even 
some  of  this  information  simultaneously  in  a  single  display,  without  overwhelming  a  viewer’s 
ability  to  make  sense  of  the  resulting  images.  This  is  partly  a  challenge  for  the  designers  of 
visualization  systems  and  partly  a  challenge  for  their  users,  the  practitioners  in  the  field,  who 
have  the  final  responsibility  for  configuring  their  visualization  systems  to  develop  meaningful 
mappings.  Unfortunately,  mapping  techniques  are  not  always  simple  to  understand  or  apply. 
Practitioners  are  now  faced  with  visualization  tools  that  offer  an  extensive  set  of  options  to 
present  information,  but  no  assistance  on  how  to  harness  or  control  the  use  of  these  options  to 
produce  an  effective  result.  As  has  been  discussed  previously  in  this  report,  the  development  of 
these  mappings  is  a  skill  that  requires  training. 

In  previous  work  (Healey,  St.  Amant,  and  Elhaddad,  1999),  we  have  designed  a 
visualization  assistant,  a  combination  of  perceptual  guidelines  and  an  intelligent  search  engine 
designed  to  identify  the  data-feature  mappings  that  are  most  appropriate  for  a  particular  dataset 
and  associated  analysis  tasks.  Our  visualization  assistant,  called  ViA,  is  specifically  engineered 
to  satisfy  a  number  of  important  goals: 

•  visually  effective:  each  mapping  suggested  by  ViA  should  produce  displays  that  allow  a 
viewer  to  rapidly,  accurately,  and  effectively  conduct  their  exploration  and  analysis, 

•  domain  independent:  ViA  should  not  be  constrained  to  a  particular  type  of  environment, 
rather,  it  should  generalize  to  a  wide  range  of  real-world  applications  including  military 
operations  planning  and  command  and  control, 

•  allow  context:  ViA  should  allow  a  viewer  to  add  domain-specific  context  as  necessary  to 
a  mapping,  and 

•  computationally  efficient:  ViA  should  not  perform  an  exhaustive  search  of  all  possible 
mappings,  rather,  it  should  concentrate  on  mappings  that  are  most  likely  to  produce  effective 
visualizations. 

The  result  is  a  semi-automated  system  that  can  identify  perceptually  salient  visualizations 
for  a  broad  collection  of  application  environments.  Viewers  can  describe  their  requirements,  ask 
ViA  to  locate  candidate  mappings,  and  refine  those  mappings  as  needed  to  produce  a  small 
collection  of  mappings  that  are  best  suited  to  a  given  dataset  and  analysis  task. 

ViA  was  designed  to  enable  users  to  develop  visualization  data  mappings  for  use  in 
solving  operational  analysis  problems.  The  implication,  however,  is  that  ViA  is  only  able  to 
support  visualization  users  that  are  currently  equipped  with  a  tool  that  contains  ViA.  The  same 
scientific  and  design  principles  that  enable  ViA  in  this  mode,  however,  can  be  productively 
turned  to  visualization  training,  better  equipping  visualization  users  regardless  of  the 
visualization  system  with  which  the  are  currently  manipulating. 

Mixed-initiative  search.  The  algorithms  within  ViA  are  based  on  recent  advances  in 
interactive  and  mixed-initiative  search  (Bumstein  &  McDermott  1996).  Some  forms  of  planning 
(Weld  1994)  bear  a  strong  resemblance  to  the  construction  of  good  visualizations,  in  that  both 
processes  rely  on  the  concepts  of  “flaws”  in  a  partially  complete  structure  or  process,  total  and 
partial  ordering  of  elements,  and  incremental  construction  and  evaluation.  Mixed-initiative 
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algorithms  have  been  modified  and  extended  to  support  external  advice  during  search.  This 
allows  the  evaluation  engines  to  guide  the  search  towards  perceptually  optimal  data-feature 
mappings.  It  also  allows  viewers  to  direct  the  selection  of  data  mappings  with  respect  to  context 
in  a  dataset,  or  to  include  features  that  they  deem  important.  It  will  also  allow  ViA  to  introspect 
over  its  incremental  construction  plan  and  describe  individual  decisions  to  the  trainee.  It  will 
also,  in  principle,  be  possible  to  feed  student  constructed  data  mappings  to  the  same  evaluation 
engine  for  review  and  critique. 

The  evaluation  engines  analyze  each  mapping  based  on  the  dataset’s  properties  and  a 
viewer’s  analysis  needs.  ViA  begins  by  asking  viewers  a  set  of  domain-independent  questions 
about  the  dataset.  The  particular  properties  we  identified  come  from  previous  work  on  automated 
visualization  (e.g.,  in  Bergman,  Rogowitz,  &  Treinish,  1995,  Gallop,  1994)  and  from  the 
psychophysical  experiments  used  to  study  the  perceptual  properties  of  color  and  texture. 
Specifically,  the  viewer  must  define: 

•  importance  ordering:  the  relative  importance  of  each  attribute, 

•  task:  the  analysis  task(s),  if  any,  to  be  performed  on  each  attribute, 

•  spatial  frequency:  the  spatial  frequency  of  each  attribute,  and 

•  domain  type:  whether  an  attribute’s  domain  is  continuous  or  discrete. 

Although  ViA  will  try  to  infer  some  of  the  dataset’s  properties  (e.g.,  spatial  frequency  and 
domain  type),  viewers  can  override  any  of  these  decisions. 

Viewer  interaction.  The  mixed-initiative  nature  of  the  search  allows  ViA  to  query 
viewers  about  choices  that  they  are  best  equipped  to  answer.  For  example,  discretizing  a 
continuous  attribute  can  allow  for  improved  mapping  (this  is  particularly  true  in  situations  where 
viewers  want  to  search  for  or  estimate  the  relative  number  of  a  specific  value  of  an  attribute).  If 
ViA  identifies  this  opportunity,  it  may  ask  the  viewer:  “Will  you  allow  me  to  discretize  this 
attribute  into  equal- width  ranges?”  Other  situations  can  cause  ViA  to  ask  to  rearranging  the 
relative  importance  of  closely  ordered  attributes,  or  to  ignore  certain  analysis  tasks  for  low- 
importance  attributes.  This  is  not  only  a  chance  for  ViA  to  generate  better  candidate  mappings;  it 
also  allows  viewers  to  refocus  their  priorities  based  on  results-to-date,  and  to  examine  in  more 
detail  specific  aspects  of  the  application  environment.  Indeed,  certain  initial  choices  for  the 
dataset  properties  and  analysis  tasks  may  receive  little  consideration  until  ViA  asks  for  particular 
changes  or  modifications.  Since  a  viewer’s  time  and  attention  are  scare  resources,  ViA  restricts 
its  queries  to  only  those  situations  where  obvious  improvements  may  result.  ViA  caches  viewer 
responses  to  ensure  that  the  same  (or  similar)  questions  are  not  asked  again. 

ViA  as  a  training  component.  As  described,  ViA  can  serve  a  number  of  roles  in  training 
basic  visualization  skills  including  (a)  providing  optimal  answers  to  act  as  references,  (b) 
providing  explanation  of  the  process  used  to  derive  these  optimal  answers,  (c)  evaluating  and 
critiquing  student  incremental  data  mapping  decisions  and  final  data  maps.  In  a  scaffolded 
training  environment  such  as  proposed  by  this  report,  these  features  can  be  used  in  isolation  or 
combination  and  faded  as  trainees  gain  mastery  over  specific  skills. 
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Review  of  Mixed-Initiative  Interaction  Support  for  Computer  Based  Training 

Mixed  initiative  has  been  defined  in  a  number  of  ways  to  suit  the  goals  of  different 
research  projects  including  significant  application  in  computer  based  training.  In  general,  mixed- 
initiative  systems  attempt  to  solve  problems  in  collaboration  with  users,  rather  than  acting  rigidly 
as  a  tool  that  only  processes  direct  commands  or  an  agent  with  complete  autonomy  in  pursuit  of 
the  goals  it  is  supplied.  The  concept  of  mixed  initiative  is  derived  from  work  in  natural  language 
processing,  in  particular  discourse  modeling.  In  natural  conversation,  participants  ask  and 
answer  questions,  supply  information  without  necessarily  being  prompted,  opportunistically 
inteiject  different  kinds  of  information  as  needed,  and  in  general  interact  with  each  other  as  peers 
in  their  exchange.  Mixed-initiative  systems  can  be  built  around  different  organizing  principles, 
but  all  mixed-initiative  systems  must  generally  address  the  following  issues: 

•  How  responsibility  is  divided  between  the  user  and  the  system  for  tasks  that  need  to  be 
performed 

•  Under  which  conditions  initiative  (in  many  cases  control)  shifts  between  the  user  and  the 
system 

•  How  a  shared  representation,  which  may  be  implicit  on  the  human  side,  can  be  shared 
between  the  user  and  the  system 

•  How  communication  proceeds  via  different  possible  genres  of  interaction 

Over  the  past  decade  or  so,  two  different  genres  of  interaction  have  dominated  research 
on  mixed  initiative:  conversational  interaction  and  direct  manipulation  interaction.  These  genres 
are  not  necessarily  disjoint;  they  can  be  combined  in  a  variety  of  ways  in  multimodal  and 
multimedia  systems  to  meet  the  requirements  of  specific  domains  and  audiences. 

Conversational  interfaces  interact  with  users  either  via  speech  or  typed  input;  they  mimic 
natural  interaction  between  human  conversants.  In  conversational  interfaces,  turn  taking  is 
common,  with  shifts  in  initiative  corresponding  to  which  participant  is  speaking  or  contributing 
to  the  conversation  at  a  given  point.  Researchers  generally  distinguish  between  dialog  initiative 
and  problem-solving  initiative.  For  example,  if  one  participant  is  asking  questions  in  order  to 
solve  a  problem  for  which  the  second  participant  has  critical  knowledge,  then  the  second 
participant,  in  guiding  the  first,  maintains  problem-solving  initiative,  while  first  participant  often 
must  take  dialog  initiative  by  prompting  for  answers. 

In  direct  manipulation  interfaces,  users  see  and  manipulate  information  through  the 
medium  of  a  virtual  graphical  environment,  such  as  the  Windows  desktop.  Interaction  with  a 
direct  manipulation  interface  is  typically  constrained  such  that  the  only  information  that  the 
system  provides  the  user  is  that  which  is  explicitly  requested  by  the  user  (neglecting  warnings 
and  alerts.)  The  drawback  to  such  interaction  is  that  if  the  system  has  information  or  capabilities 
of  which  the  user  is  unaware,  especially  in  time-sensitive  scenarios,  it  may  not  come  to  light. 
Mixed-initiative  extensions  to  direct  manipulation  interfaces  address  this  problem  by  allowing 
the  system,  under  some  circumstances,  to  autonomously  volunteer  information  or  actions  that  it 
can  perform.  One  advantage  of  this  type  of  mixed-initiative  interface  is  that  such  interfaces  can 
often  be  constrained  to  fit  into  existing  task  environments  in  which  conversational  interaction 
would  be  impractical. 
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A  number  of  different  techniques  have  been  developed  to  enable  systems  to  interact  with 
users  in  mixed-initiative  fashion,  across  both  genres  described  above.  As  with  interaction 
genres,  these  techniques  may  be  combined. 

•  Probabilistic  and  decision  theoretic  approaches:  In  early  work,  Biswas  and  Yu  (1989) 
took  an  approach  to  designing  mixed  initiative  systems  that  relied  on  the  Dempster-Shafer 
evidence  combination  scheme,  implemented  in  the  form  of  a  rule  network.  More  recently, 
Horvitz  and  colleagues  (Horvitz,  Breese,  Heckerman,  Hovel,  and  Rommelse,  2003)  have 
approached  the  problem  of  mixed  initiative  using  Bayesian  reasoning  concepts.  By  determining 
probability  distributions  over  possible  user  goals,  a  system  is  able  to  reason  about  actions  that  it 
can  take  to  facilitate  goal  achievement.  Horvitz  (1999)  has  formalized  this  work  in  the  form  of 
interface  design  principles  for  mixed-initiative  systems. 

•  Planning  approaches:  Some  of  the  best-known  results  in  mixed  initiative  come  from  the 
planning  literature.  The  TRAINS  and  TRIPS  (The  Rochester  Interactive  Planning  System) 
systems  (Ferguson,  Allen,  Miller,  Ringger,  1996,  Sikorski,  Allen,  1996)  interact  with  users 
through  a  multimedia  interface  (graphics,  speech,  text,  etc.)  to  solve  resource  allocation  problems 
collaboratively.  The  key  idea  that  such  systems  apply  is  that  human  activity  in  complex  domains 
is  often  amenable  to  partial  solution  by  planning  techniques.  A  system  that  can  offer  to  elaborate 
partial  plans  supplied  by  users,  and  possibly  to  recognize  users’  goals  based  on  their  actions,  can 
significantly  improve  overall  performance. 

•  Discourse-based  approaches:  Mixed-initiative  conversational  systems  draw  on  a  long 
history  of  research  on  the  properties  of  discourse,  which  provides  a  strong  theoretical  framework 
for  what  actions  a  system  might  take  in  service  of  shared  discourse  goals.  The  most  mature 
research  in  this  area  is  Collagen  (Lesh,  Marks,  Rich,  Sidner,  2004).  By  making  inferences  about 
the  user's  activities,  Collagen  can  ask  fewer  questions;  by  maintaining  a  model  of  user  activity. 
Collagen  can  ask  questions  at  appropriate  times  and  in  the  correct  informational  context. 

Dating  back  to  the  earliest  mixed-initiative  systems  research,  a  primary  application  area 
for  mixed-initiative  systems  has  been  computer  based  training.  Carbonell’s  SCHOLAR  system 
(Carbonell,  1971),  for  example  maintained  a  dialog  with  students  through  their  instruction  both 
providing  instructional  feedback  and  allowing  the  student  to  ask  questions.  This  approach 
continues  to  be  a  basis  for  training  systems,  including  intelligent  tutoring  system  research 
(Anderson,  Corbett,  Koedinger,  &  Pelletier,  1995)  and  user-adaptive  learning  environments 
(Meier,  Melis,  Pollet  2003).  Mixed-interaction  offers  education  systems  training  mechanisms 
that  can  both  personalize  training  and  allow  a  variety  of  student-system  dialogs  including 
feedback,  questions,  and  self-explanations. 

Summary  of  Phase  I  Results 

Our  effort  to  develop  concrete  training  goals,  methods,  and  approaches  for  battlefield 
visualization  achieved  the  following  results: 

1 .  A  working  scenario  was  defined  on  which  to  determine  project  feasibility.  This  scenario 
centered  on  a  company  level  MOUT  cordon  and  search  mission.  With  SME  support,  we 
developed  a  working  scenario  that  defined  a  tactical  environment  and  a  series  of  important 
mission  considerations  for  the  commander.  These  considerations  were  expanded  to  highlight 
information  required  for  maintaining  S  A  and  battlefield  awareness. 
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2.  A  comprehensive  GDTA  of  a  company  commander’s  information  needs  to  maintain 
situation  awareness  during  a  MOUT  Cordon  and  search  mission.  We  used  an  open  interview 
process  with  a  military  SME  to  systematically  expand  on  the  information  requirements  identified 
during  the  scenario  definition  period.  The  GDTA  that  resulted  begins  with  seven  high-level  goal 
areas  ranging  such  “Protect  the  force  /  Avoid  Casualties”  and  “Control  Civilian  Population.” 
These  high-level  goals  are  expanded  into  almost  one  hundred  individual  goals,  two  hundred 
specific  information  requirements  and  correlated  sets  of  information  types. 

3.  Review  and  synthesis  of  appropriate  literatures.  We  developed  a  description  of  basic 
visualization  skills,  built  on  current  research  in  perceptual  and  cognitive  psychology,  that 
underlie  the  connection  of  internal  cognitive  schema  and  external  visual  representations.  We 
mapped  this  set  of  skills  to  the  requirements  of  situation  awareness  and  battlespace  visualization. 
We  describe  a  number  of  historic  approaches  to  educational  pedagogy  that  have  guided  the 
development  of  training  technology  and  propose  specific  approaches  to  digital  training  of  the 
basic  visualization  skills. 

4.  Developed  a  basic  visualization  capability  for  perceptual  display  optimization  for  MOUT 
environments.  While  the  visualization  prototype  does  not,  in  its  Phase  I  state,  include 
appropriate  tools  and  content  for  training,  it  provides  a  basis  visualization  capacity  and  highlights 
the  kind  of  external  battlefield  visualizations  that  a  trainee  might  develop. 

5.  Develop  concrete  phase  II  training  goals  and  training  tool/environment  approaches.  We 
developed  and  described  four  different  training/tool  concepts  including:  an  individual  trainee 
basic  skills  part-task  trainer  and  training  module,  an  individual  trainee  whole-task  visualization 
tool  and  training  module,  an  individual  trainee  simulation  based  4D  visualization  tool,  and  a 
instructor  MOUT  visualization  training  presentation  tool. 


Phase  II  System  Design 

The  purpose  of  this  section  is  to  provide  a  high  level  overview  of  the  design  of  the 
software  infrastructure  that  will  be  built  for  Phase  II  of  the  MAVEN  project.  During  the 
MAVEN  Phase  I  we  developed  four  options  for  Phase  II  research  and  development.  The  first 
three  we  framed  loosely  around  SA  levels  and  focused  on  developing  a  training  module  and 
interactive  training  tool  to  be  used  by  individual  trainees.  The  fourth  option  focused  on 
developing  a  training  module  and  presentation  tool  to  be  used  by  a  classroom  instructor. 
Following  are  brief  descriptions  of  the  tool  concepts.  Each  concept  put  an  emphasis  on  different 
approaches  to  use  educational  technology  to  improve  battlespace  visualization  training.  While 
we  felt  each  option  has  merit  and  would  provide  benefit  to  the  Army,  we  decided  to  focus  on 
Option  A  “Basic  Visualization  Skills.” 

Option  A.  Basic  Visualization  Skills  (SA-1) 

This  option  focuses  on  developing  part-task  visualization  training  lessons  for  individual 
trainees  that  develop  basic  SA-1 -relevant  skills;  such  as  representation  reference  mapping, 
pattern  detection,  scene  change  detection,  and  other  skills  necessary  for  situational  assessment 
and  perception  of  relevant  data.  This  approach  would  use  the  underlying  visualization  system  to 
provide  basic  visualization  support  and  as  the  basis  for  a  tutorial  dialog  with  the  trainee.  This 
dialog  would  interactively  explain  visualization  rational,  suggesting  visualization  configurations, 
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and  providing  problem-solving  hints.  This  interaction  would  be  structured  as  an  educational 
scaffold,  allowing  specific  supports  to  be  removed  as  trainee  skill  and  confidence  increases.  The 
goal  of  this  scaffolded  approach  is  to  enable  the  trainee  to  develop  transferable  skills  that  will 
serve  the  trainee  regardless  of  the  visual  media  with  which  they  are  working. 

Training  focus.  This  option  stresses  individual  student  interaction  with  the  training  tool 
in  the  context  of  a  larger  military  training  course  such  as  the  Intelligence  Preparation  of  the 
Battlefield  lesson  from  Army  Training  Circular  TC  7-98-1  “Stability  and  Support  Operations 
Training  Support  Package”  (Department  of  the  Army,  1998).  During  this  interaction,  the  student 
would  be  presented  with  a  series  of  training  exercises  that  would  develop  basic  external-artifact 
based  visualization  skills,  including: 

•  Visual  Image  Comprehension:  Develop  basic  rules  of  visual  language  by  presentation 
and  examination  of  standard  military  visual  representations  (such  as  Course  of  Action  map) 
focusing  on  what  is  expressible  by  and  interpretable  from  these  representations  and  visual 
representations  in  general.  For  example,  standard  MIL-SPEC-2525b  C2  symbology  (U.S. 
Department  of  Defense,  1999)  can  represent  locations  of  enemy  units,  including  a  range  of  unit 
characteristics,  but  cannot  (by  itself)  represent  the  degree  of  certainty  in  the  assessment  of  unit 
type  or  time  since  last  verification  of  unit  position. 

•  Visual  Image  Development:  Develop  basic  rules  of  visual  image  construction  focusing 
on  translating  a  specific  information  or  communication  need  into  selection  of  appropriate 
visualization  representations  or  projections  based  on  the  characteristics  of  their  visual  language. 
For  example,  choosing  between  2D  and  3D  images  and  choosing  appropriate  display  layers  (built 
features,  terrain  features)  and  layer  configurations  requires  understanding  what  which 
representation  feature  are  required  to  articulate  a  particular  concept  or  intent.  For  example, 
Intelligence  Preparation  of  the  Battlefield  and  CCIR  recognition  have  different  requirements  for 
representing  sensor  capabilities  and  coverage. 

•  Pattern  and  Situation  Identification:  Develop  basic  skills  in  techniques  selection, 
alignment,  and  annotation  to  create  frames  of  reference  across  multiple  images  or  data  sets 
relative  to  a  particular  area  of  interest.  For  example,  using  a  time  series  of  satellite  images  to 
identify  newly  constructed  barricades,  sniper  positions,  or  possible  Improvised  Explosive  Device 
(IED)  locations. 

Research  focus.  Expansion  of  VIA  mixed-initiative  to  model,  suggest,  and/or  criticize 
student  visualization  manipulations. 

Display  mockup.  The  following  training  system  screen  interface  mockup  (see  Figure  19) 
highlights  Intelligence  Preparation  for  the  Battlefield  for  Countersniper  Measures  training  and 
suggests  what  the  basic  skills  trainer  tool  might  look  like. 


47 


Figure  19.  Basic  Skills  Trainer  Mockup,  showing  training  dialog,  display  control  and  current  tasks. 


The  primary  screen  modules  shown  include  displays  for  the  current  training  module  and 
current  task  progress,  the  mixed-initiative  training  dialog,  the  visualization  construction  panel, 
and  the  visual  display  work  area.  This  approach  balances  the  text  based  hint  and  query  capability 
of  the  training  dialog  with  the  direct  display  manipulation  of  the  visualization  construction  kit. 

Option  B.  Comprehensive  Visualization  Skills  (SA-2) 

This  option  focuses  on  developing  training  for  Situation  Awareness  Level-2  situational 
comprehension  skills.  The  technical  focus  would  be  on  developing  a  scene  creation  tool  and 
improved  visualizations  for  MOUT  environments.  Where  option  A  would  include  developing 
individual  lessons  and  encoding  them  within  the  tool,  Option  B  would  depend  on  the  trainer  to 
develop  training  problems  that  a  trainee  would  use  the  tool  to  help  solve.  This  option  also 
focuses  on  scenario  level  training  as  opposed  to  part-task  training.  For  training  we  would 
emphasize  developing  skills  necessary  to  build  internal  schema  necessary  to  conceptualize  the 
situation,  which  would  then  form  the  basis  for  situation  awareness  in  an  operational  setting. 

Training  focus.  This  option  also  stresses  individual  student  interaction  with  the  training 
tool.  During  this  interaction,  the  student  would  be  presented  with  a  series  of  training  exercises 
that  would  develop  advanced  battlefield  visualization  skills.  These  skills  would  focus  on  using 
visual  representation  to  support  specific  training  house  objectives,  such  as  using  visualization  to: 

•  Improve  sensor  placement  or  usage  for  Intelligence  Preparation  of  the  Battlefield 

•  Improve  METT-TC  awareness  for  COA  development 

Research  focus.  Expansion  of  VIA  visualization  optimization  capabilities  to  handle 
wider  range  of  Army  visual  representations  and  representation  requirements. 


48 


nation  Alignment  -  Lessen 


Current 

Goal 


Visual 

Display 

Work 

Area 


Visualization 

Construction 


Visualization 
Image  Library 


Figure  20.  Situation  awareness  trainer  mockup,  highlighting  visual  image  integration  and 
alignment. 
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Figure  21.  Situation  awareness  trainer  mockup,  highlighting  sniper  line  of  sight  development. 
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The  primary  screen  modules  shown  include  displays  for  the  primary  visualization  display  area, 
visual  image  library  and  area  map,  as  well  as  a  visualization  construction  tools  panel,  and  a 
current  goal  selection  panel. 

Option  C.  Projective  Visualization  Skills  (SA-3) 

This  option  focuses  on  developing  skills  necessary  for  SA-3,  situational  projection.  This 
option  was  not  explored  in  depth  and  was  described  as  a  known  out-of-scope  option  that  could 
build  on  A  and/or  B  at  a  later  time.  The  technical  focus  for  this  option  would  be  to  link  to  SAF- 
level  simulation  environments  that  would  allow  full  scenarios  to  be  played  out  with  adversaries, 
and  under  more  realistic  conditions.  Other  possible  features  would  include  a  full  trainer  GUI  for 
composing,  running,  and  evaluating  training  scenarios,  and  intelligent  tutoring  agents  to  assist 
the  user  in  understanding  the  training  objectives. 

Option  D.  Trainer  Development  and  Teaching  Tool 

This  option  focuses  on  developing  teaching  aids  that  could  be  used  by  trainers  to 
improve,  simplify,  and  otherwise  assist  in  their  development  and  presentation  of  MOUT 
visualization  training  materials.  It  would  include  aspects  of  each  of  the  other  three  options 
including  support  for  developing  individual  scenes  that  could  be  displayed  individually  or 
chained  together.  This  tool  is  analogous  to  Microsoft  PowerPoint  ®  for  MOUT  visualization 
training  and  would  make  use  of  an  underlying  visualization  modeling  system  as  Option  B,  but 
would  not  require  a  Trainee  interface.  Unlike  a  generic  3-D  modeling  system,  it  would  know 
about  MOUT  domain-specific  objects,  relationships,  etc.,  and  would  include  a  set  of  heuristics 
for  displaying  visual  features  of  interest. 

Training  focus.  This  option  stresses  trainer  preparation  and  presentation  of  course 
material.  This  tool  would  allow  a  course  designer  to  prepare  a  set  of  MOUT  images  based  off 
either  satellite  images  or  map  data,  annotate  these  images  with  text,  symbols,  or  specific 
relationships  markers  or  color/feature  encodings.  The  course  instructor  would  then  be  able  to 
display  and  manipulate  these  images  in  real-time.  Envisioned  manipulations  include  2D  or  3D 
zooming  and  panning  to  provide  image  fly-over  or  fly-through,  limited  animation  to  highlight 
regions  of  interest,  real-time  annotation.  We  believe  that  there  is  a  lot  of  merit  in  option  D  in  that 
it  considers  better  how  the  tool  might  be  incorporated  within  the  Officers  Advanced  Course 
training  regimen. 

Research  focus.  Design  of  visualization/scenario  specification  and  display  approach  that 
will  support  schoolhouse  visualization  training. 


50 


Figure  22.  Classroom  presentation  tool. 
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